-- Logs begin at Fri 2022-08-26 14:52:36 UTC, end at Fri 2022-08-26 16:18:16 UTC. -- Aug 26 15:34:15.521244 dv-cinder-851449-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 26 15:34:15.531322 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 26 15:34:15 2022] *** Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: nodename: dv-cinder-851449-7-iscsi Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: machine: x86_64 Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: clock source: unix Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: pcre jit disabled Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: detected number of CPU cores: 4 Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: current working directory: / Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: detected binary path: /usr/bin/uwsgi-core Aug 26 15:34:15.533214 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: your processes number limit is 47680 Aug 26 15:34:15.533805 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: your memory page size is 4096 bytes Aug 26 15:34:15.533805 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: detected max file descriptor number: 2048 Aug 26 15:34:15.533805 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: lock engine: pthread robust mutexes Aug 26 15:34:15.533805 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: thunder lock: enabled Aug 26 15:34:15.533805 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 26 15:34:15.533805 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 26 15:34:15.574125 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: Python main interpreter initialized at 0x55a8d35219c0 Aug 26 15:34:15.574125 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: python threads support enabled Aug 26 15:34:15.574125 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: your server socket listen backlog is limited to 100 connections Aug 26 15:34:15.574125 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: your mercy for graceful operations on workers is 90 seconds Aug 26 15:34:15.574482 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: mapped 403077 bytes (393 KB) for 2 cores Aug 26 15:34:15.574638 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: *** Operational MODE: preforking *** Aug 26 15:34:15.574839 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: *** uWSGI is running in multiple interpreter mode *** Aug 26 15:34:15.574839 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: spawned uWSGI master process (pid: 88497) Aug 26 15:34:15.575066 dv-cinder-851449-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 26 15:34:15.576736 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: spawned uWSGI worker 1 (pid: 88508, cores: 1) Aug 26 15:34:15.576736 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: spawned uWSGI worker 2 (pid: 88509, cores: 1) Aug 26 15:34:15.576736 dv-cinder-851449-7-iscsi devstack@keystone.service[88497]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 26 15:34:20.012245 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 15:34:20.012245 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: from cryptography.utils import int_from_bytes Aug 26 15:34:20.013131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 15:34:20.013131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: from cryptography.utils import int_from_bytes Aug 26 15:34:20.027546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 15:34:20.027546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: from cryptography.utils import int_from_bytes Aug 26 15:34:20.028399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 15:34:20.028399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: from cryptography.utils import int_from_bytes Aug 26 15:34:20.069054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 15:34:20.069257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 15:34:20.069414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] command line args: [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 15:34:20.069682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 15:34:20.069858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ================================================================================ {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 15:34:20.070112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] admin_token = **** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.070355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] config_dir = [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.070542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.070722 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] config_source = [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.071570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] debug = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.071909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.073099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.073099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.073099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.073415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.073415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] list_limit = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.073623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] log_config_append = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.073732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.073880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] log_dir = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.074056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] log_file = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.074293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.074542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.074787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.075063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.075242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.075416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.075662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.075911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.076146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.076380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.076612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.076843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.077089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.077334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.077611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.078136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.94/identity {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.078298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] publish_errors = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.078535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.078768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.078972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.079229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.079458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.079658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 15:34:20.079757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] standard_threads = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.079858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 15:34:20.079937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.080055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] command line args: [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 15:34:20.080141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.080330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.080422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 15:34:20.080500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] use_journal = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.080576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ================================================================================ {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 15:34:20.080652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] use_json = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.080729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] admin_token = **** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.080844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] use_stderr = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.080955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] use_syslog = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.081033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] config_dir = [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.081145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.081223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.081330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] config_source = [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.081437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.081515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] debug = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.081678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.081759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.081939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.082028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.082106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.082182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.082358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.082435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.082605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.082688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.082837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] list_limit = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.082935 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.083044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] log_config_append = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.083151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.083284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.083389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] log_dir = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.083465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.083566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] log_file = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.083642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.083800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.083888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.084048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.084125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.084296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.084385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.084552 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.084633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.084733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.084811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.084913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.085032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.085159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.085272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.085404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.085528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.085638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.085785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.085869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.086032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.086110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.086270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.086423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.086559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.087265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.087427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.087515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.087708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.087929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.088279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.088420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.94/identity {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.088523 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.088608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] publish_errors = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.088763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.088840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.089001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.089095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.089213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.089299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.089461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.089544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.089696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.089780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.089932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] standard_threads = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.090029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.090165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.090242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.090414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.090505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.090578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.090654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.090756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] use_journal = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.090891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.090991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] use_json = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.091109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] use_stderr = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.091182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.091288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] use_syslog = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.091374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.091553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 15:34:20.091673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.091815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.091894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092290 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.092986 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.093064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.093235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.093332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.093469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.093614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.093695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.093767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.093924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.094000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.094159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.094238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.094395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.094472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.094644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.094720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.094877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.095024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.095140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.095279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.095378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.095535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.095649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.095776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.095880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.096988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.097130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.097207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.097397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.097518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.097654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.097773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.097887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.097967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.098124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.098202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.098356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.098480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.098589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.098669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.098827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.098915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.099102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.099181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.099356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.099450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.099590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.099669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.099824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.099905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.100061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.100147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.100309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.100386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.100548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.100665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.100790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.100887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.101979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.102062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.102301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.102426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.102713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.102799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.102981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.103163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.103246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.103444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.103525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.103691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.103768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.103929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104009 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.104962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.105121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.105199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.105360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.105515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.105591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.105668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.105814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.105898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.106047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.106135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.106278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.106368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.106507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.106585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.106747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.106826 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.107983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.108061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.108226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.108305 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.108458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] auth.external = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.108544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.108691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.108769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.108959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.109036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.109211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.109301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.109444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] auth.password = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.109522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.109676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] auth.token = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.109753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.109917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.110011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.110153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.110233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.110399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.110476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.110633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.110718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.110871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] credential.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.111981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.112123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.112285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.112410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.112536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.112663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.112791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.112791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.113950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.114070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.114238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.115174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.115258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.119711 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.119925 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] federation.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.120902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.121969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] auth.external = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] auth.password = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.122993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] auth.token = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123916 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.123990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124433 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] credential.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.124938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125433 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.125897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.126958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] federation.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.127963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.128938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129636 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.129927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.130071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.130153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.130329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.130417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.130568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.130647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.130801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.130879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.131068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.131203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.131330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.131410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.131564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.131642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.131813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.131899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.132061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.132143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.132314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.132393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.132550 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.132638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.132782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.132861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.133019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.133097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.133255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.133342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.134079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.134212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.134318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.134517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.134595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.134788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.134869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.135084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.135166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.135372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.135452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.135625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.135705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.135876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.135954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.136112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.136190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.136351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.136430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.136591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.136702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.136832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.136957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.137072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.137193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.137312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.137434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.137560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.137669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.137810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.137911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.138049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.138151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.138306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.138384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.138540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.138621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.138786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.138865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.139048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.139129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.139288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.139368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.139530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.139664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.139792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.139921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.140057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.140183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.140310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.140437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.140565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.140701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.140825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] resource.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.140945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.141969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.142140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.142229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.142358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.142479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.142602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.142741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.142963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.143051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.143214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] role.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.143338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.143420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] role.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.143525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.143664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.143789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.143913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.144035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.144117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.144278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.144410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.144545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.144668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.144791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.144915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.145038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.145170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.145293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.145417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.145540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.145663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.145786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.145917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.146053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.146187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.146311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.146433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.146567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.146703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.146837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.147049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.147174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.147174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.147413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.147551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.147680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.147806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.147934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.148904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.149696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.149845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.149978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.150059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.150132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.150327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.150495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.150588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.150761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.150840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] resource.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.151049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.151144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.151400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.151567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.151646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.151806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.151884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.152045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.152131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.152279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.152362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.152515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] token.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.152593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.152757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.152844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] role.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] role.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.153963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.154041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.154209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.154297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.154454 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.154536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.154701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.154780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.154966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.155061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.155207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.155331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.155443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.155521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.155688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.155775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.155970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.156052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.156202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.156290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.156445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.156526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.156694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.156775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.156945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.157024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.connection = **** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.157193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.157377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.157466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.157622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.157702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.157879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.157958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.158115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.158201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.158353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.158432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.158589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.158668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.158826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.158913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.159094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.159174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.159331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.159410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.159576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.159668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.159812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.159890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.160049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.160127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.160286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.160371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.160533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.160653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.160767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.160846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.161006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.161093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.161242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.161321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.161393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88508) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 15:34:20.161465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.161701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.161942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] token.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.162184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.162435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.162676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.162973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.163209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.163451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.163729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.163992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.164254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.164493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.164799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.165041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.165294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.165574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.165812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.166060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.166331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.166595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.connection = **** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.166844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.167191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.167433 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.167677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.167918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.168158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.168399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.168642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.168888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.169129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.169378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.169617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.169869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.170110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.170358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.170723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.170997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.171240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 15:34:20.171413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88509) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 15:34:20.191929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.192173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.192366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.197836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.198104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.198285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.202033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.207222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.235722 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.236014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.238823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.239116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.300607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.305520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.356705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.356928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.362047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.362277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.519756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.519960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.520908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.521095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.550358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.550581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.550706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.550796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.550872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.551915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.563971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.564297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.589775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.589999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.590974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.591092 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88508) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.591193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.591301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88509) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 15:34:20.614065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.614263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 15:34:20.614497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.614590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.614706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 15:34:20.614789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 15:34:20.615070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.615170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.615274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 15:34:20.615365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.615548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.615811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.615891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.616104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.616375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.616779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.626591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.626866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.628556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.628647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.628726 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.628801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.628801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.628801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.634986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.636181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.637881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.638161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.638443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.638724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.639031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.639157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.639295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.639445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.639564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.639732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.639837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.640096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.640222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.640378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.640497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.640642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.640767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.640914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.641189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.641572 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.641850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.642121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.663693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.665211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.667485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.667629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.667715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.667715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.670247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.670534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.670811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.671732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.672021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.672299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.677247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.677534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.678691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.679002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.682355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.682650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.682954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.683245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.683527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.683710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.683805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.684000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.684092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.684282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.684372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.684568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.684881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.685156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.685426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.685693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.702274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.702559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.704010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.704315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.706382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.708230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.709573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.709869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.711515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.711811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.714048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.716064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.717135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.719349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.720333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.720629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.720909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.721189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.721462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.721738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.722570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.723172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.723466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.723747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.724022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.724312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.737666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.737962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.738230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.738501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.740635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.740935 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.741210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.741474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.746397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.747966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.748238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.749475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.750864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.751032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.751167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.751308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.751465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.751744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.752017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.753937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.754216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.754507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.754801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.755113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.764118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.764401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.764680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.765002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.765239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.765510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.765805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.767053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.767340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.767622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.767900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.768182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.768450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.768732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.781376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.784199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.784436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.787241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.787375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.788746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.790748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.791001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.791124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.791405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.792220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.793737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.794025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.794303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.797895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.799364 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.800872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.802274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.802406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.802603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.802886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.803188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.805254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.806054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.806367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.806606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.811686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.811976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.812256 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.814562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.814852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.815156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.819090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88509) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.819371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.819648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.819918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.820183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.820449 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.820718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.821062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.821328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.821611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.821874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.822142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.822272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88508) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 15:34:20.822419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.822558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.822695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88509) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.822835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.823055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.823324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.823645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.828249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.828249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.828249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.828249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.828249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.828249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.828788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.828788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88508) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 15:34:20.857284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a8d35219c0 pid: 88509 (default app) Aug 26 15:34:20.860881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408d68d-1f37-456a-8f8d-cd080c483b0d None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:20.861163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408d68d-1f37-456a-8f8d-cd080c483b0d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:20.861260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a8d35219c0 pid: 88508 (default app) Aug 26 15:34:20.861431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408d68d-1f37-456a-8f8d-cd080c483b0d None None] PATH_INFO: `/v3` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:20.865620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1/1] 10.222.0.94 () {50 vars in 791 bytes} [Fri Aug 26 15:34:20 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:28.735556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e692572-4143-4d0e-9237-62f86b335807 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:28.736103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e692572-4143-4d0e-9237-62f86b335807 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:28.736103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e692572-4143-4d0e-9237-62f86b335807 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:28.740361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1/2] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:28.746941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:28.747346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:28.747748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:28.820295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: INFO dbcounter [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] Registered counter for database keystone Aug 26 15:34:28.842647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG oslo_db.sqlalchemy.engines [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88509) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 15:34:28.849004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writer thread running {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 15:34:29.044113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG passlib.handlers.bcrypt [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88509) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 15:34:29.044555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG passlib.handlers.bcrypt [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88509) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 15:34:29.100314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:29.101977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:29.232784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 2/3] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 26 15:34:29.715869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e692572-4143-4d0e-9237-62f86b335807 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:29.787978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: INFO dbcounter [None req-0e692572-4143-4d0e-9237-62f86b335807 None None] Registered counter for database keystone Aug 26 15:34:29.808790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG oslo_db.sqlalchemy.engines [None req-0e692572-4143-4d0e-9237-62f86b335807 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88508) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 15:34:29.815765 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writer thread running {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 15:34:29.942339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e692572-4143-4d0e-9237-62f86b335807 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:29.947302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bf83a-b55f-46ed-9520-3c063a6b9a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:29.947944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf83a-b55f-46ed-9520-3c063a6b9a6a admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:29.948219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf83a-b55f-46ed-9520-3c063a6b9a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:29.948486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bf83a-b55f-46ed-9520-3c063a6b9a6a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:29.956844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bf83a-b55f-46ed-9520-3c063a6b9a6a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:30.012057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING keystone.common.rbac_enforcer.enforcer [None req-be6bf83a-b55f-46ed-9520-3c063a6b9a6a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 15:34:30.012401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bf83a-b55f-46ed-9520-3c063a6b9a6a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:30.022782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 2/4] 10.222.0.94 () {58 vars in 1324 bytes} [Fri Aug 26 15:34:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:34:30.029677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:30.044544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c359a-4893-4d99-9495-04beb5cbdd5e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:30.049033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c9ff8-07e7-47eb-b588-b3b767e4ffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:30.050137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c9ff8-07e7-47eb-b588-b3b767e4ffd7 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:30.050426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c9ff8-07e7-47eb-b588-b3b767e4ffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:30.050701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c9ff8-07e7-47eb-b588-b3b767e4ffd7 admin admin] PATH_INFO: `/v3/projects/27572d94c00243fdbbfd37ae021fbe77` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:30.052326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c9ff8-07e7-47eb-b588-b3b767e4ffd7 admin admin] RBAC: Authorizing `identity:get_project(project_id=27572d94c00243fdbbfd37ae021fbe77)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:30.107953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WARNING keystone.common.rbac_enforcer.enforcer [None req-281c9ff8-07e7-47eb-b588-b3b767e4ffd7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 15:34:30.108294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c9ff8-07e7-47eb-b588-b3b767e4ffd7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:30.112938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 3/5] 10.222.0.94 () {56 vars in 1207 bytes} [Fri Aug 26 15:34:30 2022] GET /identity/v3/projects/27572d94c00243fdbbfd37ae021fbe77 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:32.033197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5a00-00ec-436a-ba70-5d3dadea3b41 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:32.033770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5a00-00ec-436a-ba70-5d3dadea3b41 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:32.033770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5a00-00ec-436a-ba70-5d3dadea3b41 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:32.037854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 3/6] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:32.043530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf3878-e1da-463a-9b70-5743fd37962f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:32.043950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf3878-e1da-463a-9b70-5743fd37962f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:32.044078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf3878-e1da-463a-9b70-5743fd37962f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:32.063618 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-a1bf3878-e1da-463a-9b70-5743fd37962f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:32.065146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a1bf3878-e1da-463a-9b70-5743fd37962f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:32.073476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 4/7] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 15:34:32.340477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf5a00-00ec-436a-ba70-5d3dadea3b41 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:32.353106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf5a00-00ec-436a-ba70-5d3dadea3b41 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:32.357548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a26b-72fa-4f4e-ba4e-2d6bc45e1510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:32.358144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a26b-72fa-4f4e-ba4e-2d6bc45e1510 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:32.358426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a26b-72fa-4f4e-ba4e-2d6bc45e1510 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:32.358700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a26b-72fa-4f4e-ba4e-2d6bc45e1510 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:32.366477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448a26b-72fa-4f4e-ba4e-2d6bc45e1510 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:32.369463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448a26b-72fa-4f4e-ba4e-2d6bc45e1510 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:32.378357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 4/8] 10.222.0.94 () {58 vars in 1324 bytes} [Fri Aug 26 15:34:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:34:32.384014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf3878-e1da-463a-9b70-5743fd37962f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:32.393196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf3878-e1da-463a-9b70-5743fd37962f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:32.398091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0b1b3-7abd-4b9d-b7b7-f01404383298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:32.398284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0b1b3-7abd-4b9d-b7b7-f01404383298 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:32.398425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0b1b3-7abd-4b9d-b7b7-f01404383298 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:32.398697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0b1b3-7abd-4b9d-b7b7-f01404383298 admin admin] PATH_INFO: `/v3/users/5f7a209666c0478690d711f7d361c03b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:32.400638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0b1b3-7abd-4b9d-b7b7-f01404383298 admin admin] RBAC: Authorizing `identity:get_user(user_id=5f7a209666c0478690d711f7d361c03b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:32.402451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0b1b3-7abd-4b9d-b7b7-f01404383298 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:32.404006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 5/9] 10.222.0.94 () {56 vars in 1198 bytes} [Fri Aug 26 15:34:32 2022] GET /identity/v3/users/5f7a209666c0478690d711f7d361c03b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:34.372473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27255c-ab70-4f2f-a847-a1e30d14a413 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:34.373058 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27255c-ab70-4f2f-a847-a1e30d14a413 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:34.373148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27255c-ab70-4f2f-a847-a1e30d14a413 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:34.374459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 5/10] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:34.380145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00801310-fd7a-4639-a324-f3a2204c7f7a None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:34.380621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00801310-fd7a-4639-a324-f3a2204c7f7a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:34.380714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00801310-fd7a-4639-a324-f3a2204c7f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:34.398597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-00801310-fd7a-4639-a324-f3a2204c7f7a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:34.399980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-00801310-fd7a-4639-a324-f3a2204c7f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:34.417314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfda383-36e3-48e1-ad00-b71356dfe192 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:34.417603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfda383-36e3-48e1-ad00-b71356dfe192 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:34.417870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfda383-36e3-48e1-ad00-b71356dfe192 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:34.419227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 6/11] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:34:34.424379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:34.424653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:34.424920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:34.467934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG passlib.handlers.bcrypt [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88508) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 15:34:34.468348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG passlib.handlers.bcrypt [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88508) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 15:34:34.487292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 6/12] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:34:34.515270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:34.516861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:34.619393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 7/13] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:34:34.757496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00801310-fd7a-4639-a324-f3a2204c7f7a None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:34.782217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:34.864286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00801310-fd7a-4639-a324-f3a2204c7f7a None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:34.889677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:34.897337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c336-3b5d-46b8-a2d8-6575b3ab7151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:34.897978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c336-3b5d-46b8-a2d8-6575b3ab7151 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:34.898254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c336-3b5d-46b8-a2d8-6575b3ab7151 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:34.898561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5c336-3b5d-46b8-a2d8-6575b3ab7151 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:34.910612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5c336-3b5d-46b8-a2d8-6575b3ab7151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-cbc5c336-3b5d-46b8-a2d8-6575b3ab7151 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:34.919690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:34.920615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:34:34.921421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 7/14] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:34:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:34.927833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c336-3b5d-46b8-a2d8-6575b3ab7151 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:34.996409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47300d95-3a07-49f0-b8bf-489784539d63 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:34.999481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:35.012442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:35.030451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5c336-3b5d-46b8-a2d8-6575b3ab7151 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:35.034964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d44a1-c399-4024-a594-3d710ef82e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:35.035836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d44a1-c399-4024-a594-3d710ef82e32 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:35.036117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d44a1-c399-4024-a594-3d710ef82e32 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:35.036387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d44a1-c399-4024-a594-3d710ef82e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:35.065648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:35.066500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:35.066847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:35.067152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:35.067829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:35.068126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:35.069521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:35.087497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 8/15] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:35.088617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:35.111017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d44a1-c399-4024-a594-3d710ef82e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:35.112548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d44a1-c399-4024-a594-3d710ef82e32 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:35.189396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a6d17-a42a-4947-9403-42bb277dd390 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:35.214154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 8/16] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:34:35.219773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d44a1-c399-4024-a594-3d710ef82e32 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:35.222503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:35.225034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205437b-e8fc-4839-827f-7447ce847b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:35.225736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205437b-e8fc-4839-827f-7447ce847b53 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:35.226018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205437b-e8fc-4839-827f-7447ce847b53 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:35.226288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205437b-e8fc-4839-827f-7447ce847b53 None admin] PATH_INFO: `/v3/users/5f7a209666c0478690d711f7d361c03b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:35.227514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205437b-e8fc-4839-827f-7447ce847b53 None admin] RBAC: Authorizing `identity:get_user(user_id=5f7a209666c0478690d711f7d361c03b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:35.228987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205437b-e8fc-4839-827f-7447ce847b53 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:35.230570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 9/17] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:35 2022] GET /identity/v3/users/5f7a209666c0478690d711f7d361c03b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:35.236083 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205437b-e8fc-4839-827f-7447ce847b53 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:35.251258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:35.321634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d44a1-c399-4024-a594-3d710ef82e32 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:35.340297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205437b-e8fc-4839-827f-7447ce847b53 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:35.358051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:35.358964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:35.359244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:35.359515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:35.374220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531c4e6-5e67-483a-9871-a74e9320123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:35.374853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531c4e6-5e67-483a-9871-a74e9320123f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:35.375161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531c4e6-5e67-483a-9871-a74e9320123f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:35.375428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531c4e6-5e67-483a-9871-a74e9320123f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:35.376555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531c4e6-5e67-483a-9871-a74e9320123f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:35.378008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531c4e6-5e67-483a-9871-a74e9320123f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:35.379344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:35.380020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 10/18] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:35.380824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:35.386627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531c4e6-5e67-483a-9871-a74e9320123f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:34:35.400031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:35.401154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:34:35.401956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:34:35.401956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:35.401956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:34:35.401956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 9/19] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:34:35.406993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:35.466425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:35.470568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:35.490259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531c4e6-5e67-483a-9871-a74e9320123f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:35.513222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352e074-455a-4145-9021-4d559f835e20 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:35.526689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2ef50-6cbc-47cf-9629-56abc60e94d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:35.527324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2ef50-6cbc-47cf-9629-56abc60e94d8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:35.527595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2ef50-6cbc-47cf-9629-56abc60e94d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:35.527860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2ef50-6cbc-47cf-9629-56abc60e94d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:35.528578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2ef50-6cbc-47cf-9629-56abc60e94d8 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.domain.id=default, user.id=5f7a209666c0478690d711f7d361c03b) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:35.528874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2ef50-6cbc-47cf-9629-56abc60e94d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:35.530315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2ef50-6cbc-47cf-9629-56abc60e94d8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:35.536874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 11/20] 10.222.0.94 () {56 vars in 1322 bytes} [Fri Aug 26 15:34:35 2022] GET /identity/v3/role_assignments?user.id=5f7a209666c0478690d711f7d361c03b&scope.domain.id=default&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:35.547474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60bbec-4ab0-4e85-a1c9-2fe6609b9ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:35.548120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60bbec-4ab0-4e85-a1c9-2fe6609b9ca9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:35.548400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60bbec-4ab0-4e85-a1c9-2fe6609b9ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:35.548669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60bbec-4ab0-4e85-a1c9-2fe6609b9ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:35.549338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60bbec-4ab0-4e85-a1c9-2fe6609b9ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:35.549640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60bbec-4ab0-4e85-a1c9-2fe6609b9ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:35.551159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60bbec-4ab0-4e85-a1c9-2fe6609b9ca9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:35.575157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 10/21] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:37.673308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5598bc-378d-41ac-9800-8b2d6067f080 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:37.674260 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5598bc-378d-41ac-9800-8b2d6067f080 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:37.674721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5598bc-378d-41ac-9800-8b2d6067f080 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:37.676400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 12/22] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:37.682448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5838-b7fd-4689-8d05-7e5e315dc40d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:37.683314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5838-b7fd-4689-8d05-7e5e315dc40d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:37.683794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5838-b7fd-4689-8d05-7e5e315dc40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:37.711169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-42fb5838-b7fd-4689-8d05-7e5e315dc40d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:37.713234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-42fb5838-b7fd-4689-8d05-7e5e315dc40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:37.803853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 11/23] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:38.128704 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34244f-6a6c-410c-8b4d-576efb19343d None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.129401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34244f-6a6c-410c-8b4d-576efb19343d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.129871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34244f-6a6c-410c-8b4d-576efb19343d None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.131695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 13/24] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:38.137940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb71ee-66d5-4fb7-8d89-e963b5f4f1af None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.138473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb71ee-66d5-4fb7-8d89-e963b5f4f1af None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.138998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb71ee-66d5-4fb7-8d89-e963b5f4f1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.163269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-46cb71ee-66d5-4fb7-8d89-e963b5f4f1af None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:38.164805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-46cb71ee-66d5-4fb7-8d89-e963b5f4f1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:38.228225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:38.235417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfc376-2fb1-4897-a1e3-700436f90c1e None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.235696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfc376-2fb1-4897-a1e3-700436f90c1e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.236206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfc376-2fb1-4897-a1e3-700436f90c1e None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.237876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 14/25] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:38.251526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e86301-1fd6-47e3-a499-b598a8c4a760 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.252067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e86301-1fd6-47e3-a499-b598a8c4a760 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.252472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e86301-1fd6-47e3-a499-b598a8c4a760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.265912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 12/26] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:38.290961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-97e86301-1fd6-47e3-a499-b598a8c4a760 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:38.292167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-97e86301-1fd6-47e3-a499-b598a8c4a760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:38.309286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88150f-7c60-447f-b6ea-54fc8ad186ef None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.309867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88150f-7c60-447f-b6ea-54fc8ad186ef None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.310487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88150f-7c60-447f-b6ea-54fc8ad186ef None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.312599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 13/27] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:38.317796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc5b96-e7d5-4819-b6c9-477f974babfe None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.318429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc5b96-e7d5-4819-b6c9-477f974babfe None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.319193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc5b96-e7d5-4819-b6c9-477f974babfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.344897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-42fc5b96-e7d5-4819-b6c9-477f974babfe None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:38.346436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-42fc5b96-e7d5-4819-b6c9-477f974babfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:38.418476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 15/28] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:38.426954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e86301-1fd6-47e3-a499-b598a8c4a760 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:38.450753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 14/29] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:34:38.565916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:38.600100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e86301-1fd6-47e3-a499-b598a8c4a760 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:38.632017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc5b96-e7d5-4819-b6c9-477f974babfe None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:38.648262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:38.653533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.653533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.653533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.666785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:38.668333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:34:38.692369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:38.693857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:34:38.694868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:34:38.694868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:34:38.694868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:34:38.703560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 16/30] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:38.708788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:38.782737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc5b96-e7d5-4819-b6c9-477f974babfe None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:38.791323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:38.819835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017330ad-f180-4f60-8644-9ca830e98017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:38.820822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017330ad-f180-4f60-8644-9ca830e98017 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.821856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017330ad-f180-4f60-8644-9ca830e98017 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.822321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017330ad-f180-4f60-8644-9ca830e98017 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.843512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017330ad-f180-4f60-8644-9ca830e98017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-017330ad-f180-4f60-8644-9ca830e98017 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:38.855012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:38.855871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:38.856589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:34:38.860394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 15/31] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:34:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:38.861006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017330ad-f180-4f60-8644-9ca830e98017 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:38.887331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa4269-eb59-413a-bf34-56022ccdb0e3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:38.923336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:38.937339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:38.937967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:38.939935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:38.939935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:38.939935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:38.939935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:38.941079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:38.961269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 17/32] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:38.962598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:39.004365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017330ad-f180-4f60-8644-9ca830e98017 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:39.045123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954778f3-943b-42b8-9973-ac38460c4df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:39.046007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954778f3-943b-42b8-9973-ac38460c4df8 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.046450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954778f3-943b-42b8-9973-ac38460c4df8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.046860 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954778f3-943b-42b8-9973-ac38460c4df8 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.047693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954778f3-943b-42b8-9973-ac38460c4df8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:39.049254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954778f3-943b-42b8-9973-ac38460c4df8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:39.092366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aaabd-977d-4a64-8088-4da20961c8a9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:39.136344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:39.136979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.137272 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.137572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.138244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:39.138547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:39.140026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:39.155140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 18/33] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:39.155140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:39.222595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:39.282629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc47bd4-da48-487e-8f60-7a498b0dc070 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:39.314745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 16/34] 10.222.0.94 () {60 vars in 1131 bytes} [Fri Aug 26 15:34:38 2022] POST /identity/v3/roles => generated 219 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:39.316286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954778f3-943b-42b8-9973-ac38460c4df8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:39.323778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ae108-40ed-440c-84fb-a1a65be92ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:39.324839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ae108-40ed-440c-84fb-a1a65be92ada None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.325411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ae108-40ed-440c-84fb-a1a65be92ada None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.325883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ae108-40ed-440c-84fb-a1a65be92ada None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.326798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ae108-40ed-440c-84fb-a1a65be92ada None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:39.328568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ae108-40ed-440c-84fb-a1a65be92ada None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:39.359205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:39.454367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954778f3-943b-42b8-9973-ac38460c4df8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:39.496860 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d65d27-378b-42de-ad51-8ea17b808c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:39.497408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d65d27-378b-42de-ad51-8ea17b808c24 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.498106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d65d27-378b-42de-ad51-8ea17b808c24 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.498106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d65d27-378b-42de-ad51-8ea17b808c24 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.498863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d65d27-378b-42de-ad51-8ea17b808c24 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:39.500510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d65d27-378b-42de-ad51-8ea17b808c24 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:39.542266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 19/35] 10.222.0.94 () {60 vars in 1131 bytes} [Fri Aug 26 15:34:39 2022] POST /identity/v3/roles => generated 225 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:39.543199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ae108-40ed-440c-84fb-a1a65be92ada None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:39.625902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 17/36] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:34:39 2022] POST /identity/v3/projects => generated 301 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:39.628855 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae355a-f6f2-4d5c-ab49-db8d6b445773 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.629475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae355a-f6f2-4d5c-ab49-db8d6b445773 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.629956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae355a-f6f2-4d5c-ab49-db8d6b445773 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.633329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 18/37] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:39.636080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0a7c0-f9b6-4755-8ea9-9a769d7fdba7 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.640230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0a7c0-f9b6-4755-8ea9-9a769d7fdba7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.640230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0a7c0-f9b6-4755-8ea9-9a769d7fdba7 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.640230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 19/38] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:39.641361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e9e2f-1538-40e9-a809-8be697b789e5 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.641817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e9e2f-1538-40e9-a809-8be697b789e5 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.642260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e9e2f-1538-40e9-a809-8be697b789e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.670492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b96e9e2f-1538-40e9-a809-8be697b789e5 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:39.673396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b96e9e2f-1538-40e9-a809-8be697b789e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:39.679786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ae108-40ed-440c-84fb-a1a65be92ada None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:39.692550 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:39.725157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc976f6-d821-4048-beda-734f539c0610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:39.726313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc976f6-d821-4048-beda-734f539c0610 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.727102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc976f6-d821-4048-beda-734f539c0610 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.728023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc976f6-d821-4048-beda-734f539c0610 None admin] PATH_INFO: `/v3/users/5f7a209666c0478690d711f7d361c03b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.729512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc976f6-d821-4048-beda-734f539c0610 None admin] RBAC: Authorizing `identity:get_user(user_id=5f7a209666c0478690d711f7d361c03b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:39.732160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc976f6-d821-4048-beda-734f539c0610 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:39.733085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 20/39] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:39 2022] GET /identity/v3/users/5f7a209666c0478690d711f7d361c03b => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:39.737706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec25d-d73a-42fb-9e59-07fc85f5ac85 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.737706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec25d-d73a-42fb-9e59-07fc85f5ac85 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.737706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec25d-d73a-42fb-9e59-07fc85f5ac85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.762299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6ceec25d-d73a-42fb-9e59-07fc85f5ac85 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:39.762299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6ceec25d-d73a-42fb-9e59-07fc85f5ac85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:39.786637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 20/40] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:39.787929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e9e2f-1538-40e9-a809-8be697b789e5 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:39.803917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:39.855770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 21/41] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:39.910410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e9e2f-1538-40e9-a809-8be697b789e5 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:39.946413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc6861-4fd5-457f-b9db-3554ebf3c0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:39.947030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc6861-4fd5-457f-b9db-3554ebf3c0c8 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:39.947308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc6861-4fd5-457f-b9db-3554ebf3c0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:39.947580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc6861-4fd5-457f-b9db-3554ebf3c0c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:39.948692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc6861-4fd5-457f-b9db-3554ebf3c0c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:39.950244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc6861-4fd5-457f-b9db-3554ebf3c0c8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:39.952153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 21/42] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:39.957781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceec25d-d73a-42fb-9e59-07fc85f5ac85 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:40.061335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceec25d-d73a-42fb-9e59-07fc85f5ac85 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:40.068790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:40.074060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc6861-4fd5-457f-b9db-3554ebf3c0c8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:40.099878 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9294fe-d250-4695-b272-6d4cd8160917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:40.101161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9294fe-d250-4695-b272-6d4cd8160917 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:40.101665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9294fe-d250-4695-b272-6d4cd8160917 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:40.102161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9294fe-d250-4695-b272-6d4cd8160917 None admin] PATH_INFO: `/v3/domains/default/users/5f7a209666c0478690d711f7d361c03b/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:40.105379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9294fe-d250-4695-b272-6d4cd8160917 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5f7a209666c0478690d711f7d361c03b, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:40.107218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9294fe-d250-4695-b272-6d4cd8160917 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:40.155046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:40.178107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc6861-4fd5-457f-b9db-3554ebf3c0c8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:40.205033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 22/43] 10.222.0.94 () {58 vars in 1361 bytes} [Fri Aug 26 15:34:39 2022] PUT /identity/v3/domains/default/users/5f7a209666c0478690d711f7d361c03b/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:34:40.209859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9294fe-d250-4695-b272-6d4cd8160917 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:40.220621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a5038-8867-44a3-9cea-fc47f9b25045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:40.221317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a5038-8867-44a3-9cea-fc47f9b25045 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:40.221596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a5038-8867-44a3-9cea-fc47f9b25045 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:40.221872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a5038-8867-44a3-9cea-fc47f9b25045 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:40.222596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a5038-8867-44a3-9cea-fc47f9b25045 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:40.224074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a5038-8867-44a3-9cea-fc47f9b25045 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:40.304090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 22/44] 10.222.0.94 () {60 vars in 1131 bytes} [Fri Aug 26 15:34:40 2022] POST /identity/v3/roles => generated 223 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:34:40.451179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9294fe-d250-4695-b272-6d4cd8160917 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:40.491198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cba683-efa3-47f7-aeed-a2df7e405e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:40.491853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cba683-efa3-47f7-aeed-a2df7e405e38 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:40.492134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cba683-efa3-47f7-aeed-a2df7e405e38 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:40.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cba683-efa3-47f7-aeed-a2df7e405e38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:40.494596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cba683-efa3-47f7-aeed-a2df7e405e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:40.496106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cba683-efa3-47f7-aeed-a2df7e405e38 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:40.497755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 23/45] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:40.503599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a5038-8867-44a3-9cea-fc47f9b25045 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:40.615163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a5038-8867-44a3-9cea-fc47f9b25045 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:40.650206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c29ab-5523-4942-a47f-61e2a4fed003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:40.650810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c29ab-5523-4942-a47f-61e2a4fed003 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:40.651127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c29ab-5523-4942-a47f-61e2a4fed003 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:40.651406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c29ab-5523-4942-a47f-61e2a4fed003 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:40.652052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c29ab-5523-4942-a47f-61e2a4fed003 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:40.653585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c29ab-5523-4942-a47f-61e2a4fed003 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:40.659178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:40.726638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 23/46] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:34:40 2022] POST /identity/v3/projects => generated 312 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:42.173554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63699bac-60bd-4fa9-a2cd-c959fcaabdb4 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:42.174071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63699bac-60bd-4fa9-a2cd-c959fcaabdb4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:42.174071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63699bac-60bd-4fa9-a2cd-c959fcaabdb4 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:42.175465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 24/47] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:42.181377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf1d42-a013-48ab-8b29-1af367642608 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:42.181782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf1d42-a013-48ab-8b29-1af367642608 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:42.182044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf1d42-a013-48ab-8b29-1af367642608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:42.200540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-04cf1d42-a013-48ab-8b29-1af367642608 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:42.201742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-04cf1d42-a013-48ab-8b29-1af367642608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:42.268976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 24/48] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:42.549478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63699bac-60bd-4fa9-a2cd-c959fcaabdb4 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:42.590501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:42.659230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63699bac-60bd-4fa9-a2cd-c959fcaabdb4 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:42.693149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b9bc5-b27d-4e8d-8a58-737cbefdb30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:42.693924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b9bc5-b27d-4e8d-8a58-737cbefdb30f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:42.694207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b9bc5-b27d-4e8d-8a58-737cbefdb30f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:42.694481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b9bc5-b27d-4e8d-8a58-737cbefdb30f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:42.705453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b9bc5-b27d-4e8d-8a58-737cbefdb30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:42.708159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9dc96-706b-4955-a169-422c4253ec6d None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:42.708159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9dc96-706b-4955-a169-422c4253ec6d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:42.708346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9dc96-706b-4955-a169-422c4253ec6d None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:42.709652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 25/49] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-4a4b9bc5-b27d-4e8d-8a58-737cbefdb30f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:42.710806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:42.712365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:34:42.713157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 25/50] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:34:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:42.715998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a6a94-258f-494e-bf36-c297f732763a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:42.716281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a6a94-258f-494e-bf36-c297f732763a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:42.716555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a6a94-258f-494e-bf36-c297f732763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:42.718072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b9bc5-b27d-4e8d-8a58-737cbefdb30f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:42.734655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-138a6a94-258f-494e-bf36-c297f732763a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:42.735836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-138a6a94-258f-494e-bf36-c297f732763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:42.815671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:42.818457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 26/51] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:42.822911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b9bc5-b27d-4e8d-8a58-737cbefdb30f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:42.857521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:42.858138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:42.858421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:42.858697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:42.859397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:42.859700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:42.861090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:42.870049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 26/52] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:42.875646 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a6a94-258f-494e-bf36-c297f732763a None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:42.925396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:42.976805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a6a94-258f-494e-bf36-c297f732763a None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:43.014457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de990f-1253-44b1-997f-688ceee0ee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:43.015138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de990f-1253-44b1-997f-688ceee0ee28 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:43.015419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de990f-1253-44b1-997f-688ceee0ee28 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:43.015689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de990f-1253-44b1-997f-688ceee0ee28 None admin] PATH_INFO: `/v3/users/5f7a209666c0478690d711f7d361c03b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:43.017068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de990f-1253-44b1-997f-688ceee0ee28 None admin] RBAC: Authorizing `identity:get_user(user_id=5f7a209666c0478690d711f7d361c03b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:43.018570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de990f-1253-44b1-997f-688ceee0ee28 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:43.020464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 27/53] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:42 2022] GET /identity/v3/users/5f7a209666c0478690d711f7d361c03b => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:43.026955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:43.088317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de990f-1253-44b1-997f-688ceee0ee28 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:43.147212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8b64-017a-4a5a-b946-f8c56fedfcbe None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:43.186321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7cb85-7be4-4a93-93a0-e21905976c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:43.187020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7cb85-7be4-4a93-93a0-e21905976c07 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:43.187281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7cb85-7be4-4a93-93a0-e21905976c07 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:43.187555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7cb85-7be4-4a93-93a0-e21905976c07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:43.188756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7cb85-7be4-4a93-93a0-e21905976c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:43.190233 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7cb85-7be4-4a93-93a0-e21905976c07 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:43.191897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 27/54] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:43.198717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7cb85-7be4-4a93-93a0-e21905976c07 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:43.226138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:43.303978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de990f-1253-44b1-997f-688ceee0ee28 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:43.304513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7cb85-7be4-4a93-93a0-e21905976c07 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:43.320772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:43.338436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87834e-dd73-47b1-a6df-0a4ecca947c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:43.339040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87834e-dd73-47b1-a6df-0a4ecca947c1 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:43.339423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87834e-dd73-47b1-a6df-0a4ecca947c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:43.339591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87834e-dd73-47b1-a6df-0a4ecca947c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:43.340687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87834e-dd73-47b1-a6df-0a4ecca947c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:43.340934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12a9db-0e7b-4f10-8c3d-1e9c2f449369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:43.341614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12a9db-0e7b-4f10-8c3d-1e9c2f449369 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:43.341894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12a9db-0e7b-4f10-8c3d-1e9c2f449369 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:43.342224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12a9db-0e7b-4f10-8c3d-1e9c2f449369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:43.342312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87834e-dd73-47b1-a6df-0a4ecca947c1 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:43.342894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12a9db-0e7b-4f10-8c3d-1e9c2f449369 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.domain.id=default, user.id=5f7a209666c0478690d711f7d361c03b) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:43.343236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12a9db-0e7b-4f10-8c3d-1e9c2f449369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:43.344210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 28/55] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:43.344908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12a9db-0e7b-4f10-8c3d-1e9c2f449369 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:43.349306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87834e-dd73-47b1-a6df-0a4ecca947c1 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:43.361610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 28/56] 10.222.0.94 () {56 vars in 1322 bytes} [Fri Aug 26 15:34:43 2022] GET /identity/v3/role_assignments?user.id=5f7a209666c0478690d711f7d361c03b&scope.domain.id=default&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:43.463425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87834e-dd73-47b1-a6df-0a4ecca947c1 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:43.500692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707b12c-6b96-4349-950a-e14efebcfbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:43.501307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b12c-6b96-4349-950a-e14efebcfbc4 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:43.501583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b12c-6b96-4349-950a-e14efebcfbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:43.501855 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b12c-6b96-4349-950a-e14efebcfbc4 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:43.502501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707b12c-6b96-4349-950a-e14efebcfbc4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:43.503972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707b12c-6b96-4349-950a-e14efebcfbc4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:43.584118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 29/57] 10.222.0.94 () {60 vars in 1140 bytes} [Fri Aug 26 15:34:43 2022] POST /identity/v3/projects => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:34:45.537673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376ece8-89fe-43c1-8305-d8ce38c498a4 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:45.538224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376ece8-89fe-43c1-8305-d8ce38c498a4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:45.538224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376ece8-89fe-43c1-8305-d8ce38c498a4 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:45.539589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 29/58] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:45.544708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059b3f2-64a7-424e-bb57-68a3045a1814 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:45.545120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059b3f2-64a7-424e-bb57-68a3045a1814 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:45.545260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059b3f2-64a7-424e-bb57-68a3045a1814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:45.563291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a059b3f2-64a7-424e-bb57-68a3045a1814 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:45.564521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a059b3f2-64a7-424e-bb57-68a3045a1814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:45.595016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:45.635875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 30/59] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:45.904038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376ece8-89fe-43c1-8305-d8ce38c498a4 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:45.992719 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:46.006443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376ece8-89fe-43c1-8305-d8ce38c498a4 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:46.040770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7903106-571d-4845-8539-b3f9c68e4992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:46.041396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903106-571d-4845-8539-b3f9c68e4992 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:46.041674 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903106-571d-4845-8539-b3f9c68e4992 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:46.041945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903106-571d-4845-8539-b3f9c68e4992 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:46.043093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7903106-571d-4845-8539-b3f9c68e4992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:46.044526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7903106-571d-4845-8539-b3f9c68e4992 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:46.046084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 30/60] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:46.051744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059b3f2-64a7-424e-bb57-68a3045a1814 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:46.150112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059b3f2-64a7-424e-bb57-68a3045a1814 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:46.183686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195b31-4e9d-477e-847b-20fa2dbceef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:46.184228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195b31-4e9d-477e-847b-20fa2dbceef0 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:46.184511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195b31-4e9d-477e-847b-20fa2dbceef0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:46.184786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195b31-4e9d-477e-847b-20fa2dbceef0 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:46.185429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195b31-4e9d-477e-847b-20fa2dbceef0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:46.186876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195b31-4e9d-477e-847b-20fa2dbceef0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:46.290551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 31/61] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:34:46 2022] POST /identity/v3/users => generated 275 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:49.451302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2fb07-aa2c-4189-849d-16e04399d5f4 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:49.452587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2fb07-aa2c-4189-849d-16e04399d5f4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:49.453365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2fb07-aa2c-4189-849d-16e04399d5f4 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:49.455710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 31/62] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:49.461199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317976ff-5413-46ef-b1c5-3173cc51921b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:49.462091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317976ff-5413-46ef-b1c5-3173cc51921b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:49.462560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317976ff-5413-46ef-b1c5-3173cc51921b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:49.492229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aef57-748a-4981-a943-fe62a8e9338f None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:49.492229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aef57-748a-4981-a943-fe62a8e9338f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:49.492229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5aef57-748a-4981-a943-fe62a8e9338f None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:49.492229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 32/63] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:34:49.498520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-317976ff-5413-46ef-b1c5-3173cc51921b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:49.499515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf330-8d4a-4f44-9ac3-c100fe51837e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:49.500026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf330-8d4a-4f44-9ac3-c100fe51837e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:49.500126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-317976ff-5413-46ef-b1c5-3173cc51921b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:49.500608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbf330-8d4a-4f44-9ac3-c100fe51837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:49.538204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-32cbf330-8d4a-4f44-9ac3-c100fe51837e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:49.541414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-32cbf330-8d4a-4f44-9ac3-c100fe51837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:49.547297 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:49.650875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 32/64] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:49.685121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 33/65] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:34:49.877394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d19b1-81dd-4a85-a6b1-2fd16662c62a None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:49.877863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d19b1-81dd-4a85-a6b1-2fd16662c62a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:49.878008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d19b1-81dd-4a85-a6b1-2fd16662c62a None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:49.879603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 33/66] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:49.884772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da86b3-daae-49d3-a2f1-1bae64de4928 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:49.885063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da86b3-daae-49d3-a2f1-1bae64de4928 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:49.885475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da86b3-daae-49d3-a2f1-1bae64de4928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:49.911293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-96da86b3-daae-49d3-a2f1-1bae64de4928 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:49.912519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-96da86b3-daae-49d3-a2f1-1bae64de4928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:49.919794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae3d9c-b140-4e81-9631-18f866b140c6 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:49.920078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae3d9c-b140-4e81-9631-18f866b140c6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:49.920350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae3d9c-b140-4e81-9631-18f866b140c6 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:49.921918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 34/67] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:34:49.939394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8f6b-649a-4719-a75d-4c96889c438b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:49.939652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8f6b-649a-4719-a75d-4c96889c438b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:49.939934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8f6b-649a-4719-a75d-4c96889c438b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:49.959672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-86cd8f6b-649a-4719-a75d-4c96889c438b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:49.961148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-86cd8f6b-649a-4719-a75d-4c96889c438b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:50.001075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 34/68] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:50.002355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2118bd-88cc-4123-86dd-f12536493663 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.002638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2118bd-88cc-4123-86dd-f12536493663 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.002910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2118bd-88cc-4123-86dd-f12536493663 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.004191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 35/69] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:50.004883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2118bd-88cc-4123-86dd-f12536493663 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.028201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:50.055083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 35/70] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:50.056491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0b545-eabc-43cf-8878-f71e6324fd6f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.056775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0b545-eabc-43cf-8878-f71e6324fd6f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.057041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0b545-eabc-43cf-8878-f71e6324fd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.096661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-15f0b545-eabc-43cf-8878-f71e6324fd6f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:50.098381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-15f0b545-eabc-43cf-8878-f71e6324fd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:50.160173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2118bd-88cc-4123-86dd-f12536493663 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.184416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 36/71] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:50.184416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0b545-eabc-43cf-8878-f71e6324fd6f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.202948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:50.204575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d720b-f740-4291-987f-1821f8943802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.204575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d720b-f740-4291-987f-1821f8943802 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.204575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d720b-f740-4291-987f-1821f8943802 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.204820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d720b-f740-4291-987f-1821f8943802 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.216376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d720b-f740-4291-987f-1821f8943802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-082d720b-f740-4291-987f-1821f8943802 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:50.222684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:50.223540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:34:50.225636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 36/72] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:50.234950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d720b-f740-4291-987f-1821f8943802 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.297061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0b545-eabc-43cf-8878-f71e6324fd6f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.314160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:50.338018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afdf5f-f172-4366-afdb-d7d5a71b9ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.338655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afdf5f-f172-4366-afdb-d7d5a71b9ed0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.338947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afdf5f-f172-4366-afdb-d7d5a71b9ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.339218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afdf5f-f172-4366-afdb-d7d5a71b9ed0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.347540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d720b-f740-4291-987f-1821f8943802 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.354489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afdf5f-f172-4366-afdb-d7d5a71b9ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-48afdf5f-f172-4366-afdb-d7d5a71b9ed0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:50.361853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:50.362792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:50.363559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:34:50.366532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 37/73] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:50.367767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afdf5f-f172-4366-afdb-d7d5a71b9ed0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.389112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.389896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.390249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.390665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.391500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:50.391897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:50.393568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:50.403855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 37/74] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:50.404744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.456923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:50.494093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afdf5f-f172-4366-afdb-d7d5a71b9ed0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.519774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5b552-507c-44af-8da4-6a7d7c23cb57 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.531539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7082e-7770-4207-a485-d09303f0cdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.532180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7082e-7770-4207-a485-d09303f0cdab None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.532452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7082e-7770-4207-a485-d09303f0cdab None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.532719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7082e-7770-4207-a485-d09303f0cdab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.544294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7082e-7770-4207-a485-d09303f0cdab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-5af7082e-7770-4207-a485-d09303f0cdab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:50.548891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:50.549814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:34:50.550570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 38/75] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:50.551236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7082e-7770-4207-a485-d09303f0cdab None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.560451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.561448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.561927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.562406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.563401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:50.563834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:50.565455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:50.570966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:50.575340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 38/76] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:50.576231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.660002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7082e-7770-4207-a485-d09303f0cdab None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.681261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a5b64-5a50-4f24-8087-a46841dbb26c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.695918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:50.704052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2622983-e14b-4c7b-9fc1-fb01f3eab976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.705070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622983-e14b-4c7b-9fc1-fb01f3eab976 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.705532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622983-e14b-4c7b-9fc1-fb01f3eab976 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.705914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2622983-e14b-4c7b-9fc1-fb01f3eab976 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.724816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2622983-e14b-4c7b-9fc1-fb01f3eab976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-c2622983-e14b-4c7b-9fc1-fb01f3eab976 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:50.731357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:50.732326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:34:50.733064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:34:50.733616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dae73-0304-4c2b-a4eb-ad09b9eaf974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.733616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dae73-0304-4c2b-a4eb-ad09b9eaf974 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.733747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 39/77] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:50.734033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dae73-0304-4c2b-a4eb-ad09b9eaf974 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.734498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2622983-e14b-4c7b-9fc1-fb01f3eab976 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.734659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dae73-0304-4c2b-a4eb-ad09b9eaf974 None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.750335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dae73-0304-4c2b-a4eb-ad09b9eaf974 None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:50.751830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dae73-0304-4c2b-a4eb-ad09b9eaf974 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:50.753634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 39/78] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:50.754252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dae73-0304-4c2b-a4eb-ad09b9eaf974 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.818993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:50.844527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2622983-e14b-4c7b-9fc1-fb01f3eab976 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.864574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dae73-0304-4c2b-a4eb-ad09b9eaf974 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:50.878932 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63efea-55b6-4ae8-9365-1558af66f5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.879578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63efea-55b6-4ae8-9365-1558af66f5a3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.879849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63efea-55b6-4ae8-9365-1558af66f5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.880114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63efea-55b6-4ae8-9365-1558af66f5a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.881238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63efea-55b6-4ae8-9365-1558af66f5a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:50.882701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63efea-55b6-4ae8-9365-1558af66f5a3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:50.884282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 40/79] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:50.885044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63efea-55b6-4ae8-9365-1558af66f5a3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.900818 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:50.901440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:50.901841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:50.901991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:50.903367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:50.903916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:50.905607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:50.914264 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 40/80] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:50.915930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:50.948912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:50.991980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63efea-55b6-4ae8-9365-1558af66f5a3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.025867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed560e0-5c9c-4dc0-862d-50c3118388ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.026494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed560e0-5c9c-4dc0-862d-50c3118388ff None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.026770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed560e0-5c9c-4dc0-862d-50c3118388ff None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.027068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed560e0-5c9c-4dc0-862d-50c3118388ff None admin] PATH_INFO: `/v3/users/5f7a209666c0478690d711f7d361c03b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:51.028866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed560e0-5c9c-4dc0-862d-50c3118388ff None admin] RBAC: Authorizing `identity:get_user(user_id=5f7a209666c0478690d711f7d361c03b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:51.030305 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed560e0-5c9c-4dc0-862d-50c3118388ff None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:51.030872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a7ae-2dd1-46a7-b5f8-ac632439b976 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.031907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 41/81] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/users/5f7a209666c0478690d711f7d361c03b => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:51.032758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed560e0-5c9c-4dc0-862d-50c3118388ff None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:51.054262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:51.069405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.070022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.070301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.070574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:51.071278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:51.071580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:51.073039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:51.083816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 41/82] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:34:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:51.084635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:51.136182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed560e0-5c9c-4dc0-862d-50c3118388ff None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.163940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:51.171655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a868-2cb4-4afe-b034-9d6fcc717664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.172276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a868-2cb4-4afe-b034-9d6fcc717664 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.175942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a868-2cb4-4afe-b034-9d6fcc717664 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.175942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a868-2cb4-4afe-b034-9d6fcc717664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:51.186761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d582e6-ef91-46df-a180-1aaf89195cd0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.221628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9414270-d508-444f-becd-2f82c38e2e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.222239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414270-d508-444f-becd-2f82c38e2e92 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.222518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414270-d508-444f-becd-2f82c38e2e92 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.222790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9414270-d508-444f-becd-2f82c38e2e92 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:51.223556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9414270-d508-444f-becd-2f82c38e2e92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:51.225012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9414270-d508-444f-becd-2f82c38e2e92 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:51.244385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4a868-2cb4-4afe-b034-9d6fcc717664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:51.245868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4a868-2cb4-4afe-b034-9d6fcc717664 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:51.314145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 42/83] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:34:51 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:51.315218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9414270-d508-444f-becd-2f82c38e2e92 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:51.369141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:51.370398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 42/84] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:34:51.371905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a868-2cb4-4afe-b034-9d6fcc717664 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:51.406761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:51.446596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9414270-d508-444f-becd-2f82c38e2e92 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.491019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df3a44-7afb-4003-9137-8274b2cccaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.491678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df3a44-7afb-4003-9137-8274b2cccaf0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.491960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df3a44-7afb-4003-9137-8274b2cccaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.492245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df3a44-7afb-4003-9137-8274b2cccaf0 None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:51.493661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df3a44-7afb-4003-9137-8274b2cccaf0 None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:51.494769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a868-2cb4-4afe-b034-9d6fcc717664 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.495284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df3a44-7afb-4003-9137-8274b2cccaf0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:51.498324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 43/85] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:51 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:51.498324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df3a44-7afb-4003-9137-8274b2cccaf0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:51.547972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977938cb-d544-4f3b-b8ea-0a3f3b1a4884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.547972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977938cb-d544-4f3b-b8ea-0a3f3b1a4884 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.547972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977938cb-d544-4f3b-b8ea-0a3f3b1a4884 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.547972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977938cb-d544-4f3b-b8ea-0a3f3b1a4884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:51.639621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df3a44-7afb-4003-9137-8274b2cccaf0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.641184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977938cb-d544-4f3b-b8ea-0a3f3b1a4884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:51.642961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977938cb-d544-4f3b-b8ea-0a3f3b1a4884 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:51.648073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:51.680147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:51.690460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9d3d5-5ec6-4408-aa5c-6f785a0a6919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.691178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9d3d5-5ec6-4408-aa5c-6f785a0a6919 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.691468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9d3d5-5ec6-4408-aa5c-6f785a0a6919 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.691750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9d3d5-5ec6-4408-aa5c-6f785a0a6919 None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:51.692950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9d3d5-5ec6-4408-aa5c-6f785a0a6919 None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:51.696409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9d3d5-5ec6-4408-aa5c-6f785a0a6919 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:51.698964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 44/86] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:51 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:51.700628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9d3d5-5ec6-4408-aa5c-6f785a0a6919 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:51.775838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 43/87] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:34:51.777207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977938cb-d544-4f3b-b8ea-0a3f3b1a4884 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:51.837953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9d3d5-5ec6-4408-aa5c-6f785a0a6919 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.884913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc2bf-bab7-47e3-9a14-9bb5e74bf8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.885556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc2bf-bab7-47e3-9a14-9bb5e74bf8ad None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.885832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc2bf-bab7-47e3-9a14-9bb5e74bf8ad None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.886104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc2bf-bab7-47e3-9a14-9bb5e74bf8ad None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:51.887955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc2bf-bab7-47e3-9a14-9bb5e74bf8ad None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:51.889747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc2bf-bab7-47e3-9a14-9bb5e74bf8ad None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:51.892162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 45/88] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:51 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:51.893217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc2bf-bab7-47e3-9a14-9bb5e74bf8ad None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:51.901708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977938cb-d544-4f3b-b8ea-0a3f3b1a4884 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:51.936940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:51.943503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:51.946106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06454a-b0d7-4647-b79f-4aae3b1b04a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:51.946738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06454a-b0d7-4647-b79f-4aae3b1b04a5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:51.947066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06454a-b0d7-4647-b79f-4aae3b1b04a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:51.950237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06454a-b0d7-4647-b79f-4aae3b1b04a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.005846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc2bf-bab7-47e3-9a14-9bb5e74bf8ad None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:52.027535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06454a-b0d7-4647-b79f-4aae3b1b04a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.029252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06454a-b0d7-4647-b79f-4aae3b1b04a5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.045918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56f6ff-1d62-46a7-9841-6131b2b313b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:52.046724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f6ff-1d62-46a7-9841-6131b2b313b8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:52.047203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f6ff-1d62-46a7-9841-6131b2b313b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:52.047553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f6ff-1d62-46a7-9841-6131b2b313b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.125278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56f6ff-1d62-46a7-9841-6131b2b313b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.127117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56f6ff-1d62-46a7-9841-6131b2b313b8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.145095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 44/89] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:34:52.146072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06454a-b0d7-4647-b79f-4aae3b1b04a5 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:52.176657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:52.179617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:52.243087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 46/90] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:34:52.243989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56f6ff-1d62-46a7-9841-6131b2b313b8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:52.258748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06454a-b0d7-4647-b79f-4aae3b1b04a5 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:52.297292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878115b-8d33-4124-8e3b-fed77a1f5efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:52.297964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878115b-8d33-4124-8e3b-fed77a1f5efb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:52.298249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878115b-8d33-4124-8e3b-fed77a1f5efb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:52.298521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878115b-8d33-4124-8e3b-fed77a1f5efb None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.299774 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878115b-8d33-4124-8e3b-fed77a1f5efb None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.301450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878115b-8d33-4124-8e3b-fed77a1f5efb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.303212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 45/91] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:52 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:52.304099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878115b-8d33-4124-8e3b-fed77a1f5efb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:52.350703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56f6ff-1d62-46a7-9841-6131b2b313b8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:52.390508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:52.391239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:52.391517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:52.391790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.392634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] RBAC: Adding query filter params (role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user.id=c7bb81025bd540828ef536c262646131) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:52.392935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.394501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.400601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 47/92] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:34:52 2022] GET /identity/v3/role_assignments?user.id=c7bb81025bd540828ef536c262646131&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:52.404031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:52.413349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:52.420210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878115b-8d33-4124-8e3b-fed77a1f5efb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:52.420637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:52.462447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0442b5-1669-41da-9db1-bae882e22502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:52.463121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0442b5-1669-41da-9db1-bae882e22502 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:52.463403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0442b5-1669-41da-9db1-bae882e22502 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:52.463676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0442b5-1669-41da-9db1-bae882e22502 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.464849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0442b5-1669-41da-9db1-bae882e22502 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.466334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0442b5-1669-41da-9db1-bae882e22502 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.468370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 46/93] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:52 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:52.469213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0442b5-1669-41da-9db1-bae882e22502 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:52.537502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f4724-282b-47ec-9c64-3d7f74a6e542 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:52.585100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c53b4e-3ab3-443b-b246-203d1a6287f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:52.586055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c53b4e-3ab3-443b-b246-203d1a6287f9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:52.586482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c53b4e-3ab3-443b-b246-203d1a6287f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:52.586889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c53b4e-3ab3-443b-b246-203d1a6287f9 None admin] PATH_INFO: `/v3/projects/cf5a7cedc7e34dafa19c08c291efae39` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.588457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c53b4e-3ab3-443b-b246-203d1a6287f9 None admin] RBAC: Authorizing `identity:get_project(project_id=cf5a7cedc7e34dafa19c08c291efae39)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.590136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c53b4e-3ab3-443b-b246-203d1a6287f9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.592296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 48/94] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:52 2022] GET /identity/v3/projects/cf5a7cedc7e34dafa19c08c291efae39 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:52.593462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c53b4e-3ab3-443b-b246-203d1a6287f9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:52.608467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0442b5-1669-41da-9db1-bae882e22502 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:52.672881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:52.673513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:52.673793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:52.674065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.683329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:52.685186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user.id=5f7a209666c0478690d711f7d361c03b) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:52.685333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.686948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.701779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 47/95] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:34:52 2022] GET /identity/v3/role_assignments?user.id=5f7a209666c0478690d711f7d361c03b&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:52.702480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:52.714968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c53b4e-3ab3-443b-b246-203d1a6287f9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:52.763469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:52.832421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e8cac-5bfd-43d3-8b6d-bde67e595de9 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:52.839273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414bbb7-1bf1-4434-8566-ccff7772a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:52.840468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414bbb7-1bf1-4434-8566-ccff7772a48d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:52.840754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414bbb7-1bf1-4434-8566-ccff7772a48d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:52.841031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414bbb7-1bf1-4434-8566-ccff7772a48d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.842732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414bbb7-1bf1-4434-8566-ccff7772a48d None admin] RBAC: Adding query filter params (role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user.id=c7bb81025bd540828ef536c262646131) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:52.843080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414bbb7-1bf1-4434-8566-ccff7772a48d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.844557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414bbb7-1bf1-4434-8566-ccff7772a48d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.856772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 49/96] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:34:52 2022] GET /identity/v3/role_assignments?user.id=c7bb81025bd540828ef536c262646131&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:52.892537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091b81c-8d58-4ef7-89ed-51bf1c58cd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:52.893242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091b81c-8d58-4ef7-89ed-51bf1c58cd75 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:52.893712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091b81c-8d58-4ef7-89ed-51bf1c58cd75 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:52.893999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091b81c-8d58-4ef7-89ed-51bf1c58cd75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:52.894835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091b81c-8d58-4ef7-89ed-51bf1c58cd75 None admin] RBAC: Adding query filter params (role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=cf5a7cedc7e34dafa19c08c291efae39, user.id=c7bb81025bd540828ef536c262646131) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:52.895257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091b81c-8d58-4ef7-89ed-51bf1c58cd75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:52.896795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091b81c-8d58-4ef7-89ed-51bf1c58cd75 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:52.905336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 48/97] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:34:52 2022] GET /identity/v3/role_assignments?user.id=c7bb81025bd540828ef536c262646131&scope.project.id=cf5a7cedc7e34dafa19c08c291efae39&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:53.613315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876e307-404c-4409-8c9d-86fb9f50a344 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:53.613974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876e307-404c-4409-8c9d-86fb9f50a344 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:53.614479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876e307-404c-4409-8c9d-86fb9f50a344 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:53.616107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 50/98] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:53.622895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942b4af-abe9-4de6-887a-07359e4caf87 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:53.623797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942b4af-abe9-4de6-887a-07359e4caf87 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:53.624289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942b4af-abe9-4de6-887a-07359e4caf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:53.647048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1942b4af-abe9-4de6-887a-07359e4caf87 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:53.648589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1942b4af-abe9-4de6-887a-07359e4caf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:53.747178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 49/99] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:53.966077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ee53-615a-422f-8195-eef9a37bc384 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:53.967114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ee53-615a-422f-8195-eef9a37bc384 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:53.967621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205ee53-615a-422f-8195-eef9a37bc384 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:53.969468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 51/100] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:53.975354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93c95-4c51-4645-bd84-ee5a4369f2cb None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:53.976315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93c95-4c51-4645-bd84-ee5a4369f2cb None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:53.977111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93c95-4c51-4645-bd84-ee5a4369f2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.006361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-acb93c95-4c51-4645-bd84-ee5a4369f2cb None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:54.007967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-acb93c95-4c51-4645-bd84-ee5a4369f2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:54.026658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:54.088673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 50/101] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:54.136117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205ee53-615a-422f-8195-eef9a37bc384 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:54.262653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205ee53-615a-422f-8195-eef9a37bc384 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:54.303446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56c465-37a0-4873-880d-a721184fd601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:54.304095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56c465-37a0-4873-880d-a721184fd601 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:54.304380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56c465-37a0-4873-880d-a721184fd601 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:54.304659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56c465-37a0-4873-880d-a721184fd601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.305783 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56c465-37a0-4873-880d-a721184fd601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:54.307418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56c465-37a0-4873-880d-a721184fd601 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:54.309820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 52/102] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:54.315232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93c95-4c51-4645-bd84-ee5a4369f2cb None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:54.423202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93c95-4c51-4645-bd84-ee5a4369f2cb None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:54.460659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6385-fa04-4b34-8297-f89049bb1d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:54.461340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6385-fa04-4b34-8297-f89049bb1d76 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:54.461633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6385-fa04-4b34-8297-f89049bb1d76 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:54.461914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6385-fa04-4b34-8297-f89049bb1d76 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.462586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e6385-fa04-4b34-8297-f89049bb1d76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:54.464103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e6385-fa04-4b34-8297-f89049bb1d76 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:54.550504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 51/103] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:34:54 2022] POST /identity/v3/projects => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:54.644897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56c465-37a0-4873-880d-a721184fd601 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:54.656170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:54.766911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56c465-37a0-4873-880d-a721184fd601 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:54.807030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5aa64-cc27-4d0e-afd5-6ccf76bd9d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:54.808056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5aa64-cc27-4d0e-afd5-6ccf76bd9d15 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:54.809503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5aa64-cc27-4d0e-afd5-6ccf76bd9d15 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:54.810002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5aa64-cc27-4d0e-afd5-6ccf76bd9d15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:54.822318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5aa64-cc27-4d0e-afd5-6ccf76bd9d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-5fe5aa64-cc27-4d0e-afd5-6ccf76bd9d15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:54.828177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:54.829117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:34:54.830038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:34:54.832039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 53/104] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:54.838014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6385-fa04-4b34-8297-f89049bb1d76 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:54.843214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:54.977981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6385-fa04-4b34-8297-f89049bb1d76 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:55.014892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:55.015535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:55.015809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:55.016083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:55.016756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] RBAC: Adding query filter params (name=reader) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:55.017057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:55.018451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:55.032065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 52/105] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:34:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:55.038083 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5aa64-cc27-4d0e-afd5-6ccf76bd9d15 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:55.134622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:55.177715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5aa64-cc27-4d0e-afd5-6ccf76bd9d15 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:55.225543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796865b-663a-465e-a106-0bd3e3bb6f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:55.226191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796865b-663a-465e-a106-0bd3e3bb6f0a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:55.226472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796865b-663a-465e-a106-0bd3e3bb6f0a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:55.226743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796865b-663a-465e-a106-0bd3e3bb6f0a None admin] PATH_INFO: `/v3/users/142d81749cb3486b87b11184e2cd8b6e` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:55.250150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796865b-663a-465e-a106-0bd3e3bb6f0a None admin] RBAC: Authorizing `identity:get_user(user_id=142d81749cb3486b87b11184e2cd8b6e)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:55.251671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796865b-663a-465e-a106-0bd3e3bb6f0a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:55.253741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 54/106] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:55 2022] GET /identity/v3/users/142d81749cb3486b87b11184e2cd8b6e => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:55.259769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:55.332596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a58b7-520a-416c-a440-7a2b13cfc287 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:55.332882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a58b7-520a-416c-a440-7a2b13cfc287 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:55.333364 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a58b7-520a-416c-a440-7a2b13cfc287 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:55.334912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 55/107] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:34:55.340813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee06d-1fec-48bb-9ecf-2a8ed4bfa678 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:55.341166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee06d-1fec-48bb-9ecf-2a8ed4bfa678 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:55.341508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee06d-1fec-48bb-9ecf-2a8ed4bfa678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:55.391560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:55.394626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0abee06d-1fec-48bb-9ecf-2a8ed4bfa678 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:55.401572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0abee06d-1fec-48bb-9ecf-2a8ed4bfa678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:55.487746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa437c-14ca-406b-98ec-a993febf233a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:55.564558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d3e96-71f4-42d5-b55d-c8676c3a0dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:55.565572 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3e96-71f4-42d5-b55d-c8676c3a0dcc None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:55.566109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3e96-71f4-42d5-b55d-c8676c3a0dcc None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:55.566569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3e96-71f4-42d5-b55d-c8676c3a0dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:55.597917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 56/108] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:55.600598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d192-2671-4fb1-a27c-b45ec2d9da11 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:55.600598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d192-2671-4fb1-a27c-b45ec2d9da11 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:55.600598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d192-2671-4fb1-a27c-b45ec2d9da11 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:55.601363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 57/109] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:55.620781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384e427-eeca-4f49-bdf4-b82f6b4715d9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:55.621137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384e427-eeca-4f49-bdf4-b82f6b4715d9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:55.621424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384e427-eeca-4f49-bdf4-b82f6b4715d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:55.648736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d3e96-71f4-42d5-b55d-c8676c3a0dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:55.650514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d3e96-71f4-42d5-b55d-c8676c3a0dcc None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:55.655305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4384e427-eeca-4f49-bdf4-b82f6b4715d9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:55.656610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4384e427-eeca-4f49-bdf4-b82f6b4715d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:55.721372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:55.727657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:55.744795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 58/110] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 26 15:34:55.767832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 53/111] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:34:55.773862 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384e427-eeca-4f49-bdf4-b82f6b4715d9 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:55.921307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384e427-eeca-4f49-bdf4-b82f6b4715d9 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:55.980204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb1b59-aff6-4f6d-accc-b4ffeb166b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:55.980885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1b59-aff6-4f6d-accc-b4ffeb166b80 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:55.981170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1b59-aff6-4f6d-accc-b4ffeb166b80 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:55.981446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1b59-aff6-4f6d-accc-b4ffeb166b80 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:55.982627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb1b59-aff6-4f6d-accc-b4ffeb166b80 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:55.984150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb1b59-aff6-4f6d-accc-b4ffeb166b80 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:55.985978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 59/112] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:55 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:55.992371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d3e96-71f4-42d5-b55d-c8676c3a0dcc None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:56.095850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb1b59-aff6-4f6d-accc-b4ffeb166b80 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:56.155190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d3e96-71f4-42d5-b55d-c8676c3a0dcc None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:56.155865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:56.209570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb1b59-aff6-4f6d-accc-b4ffeb166b80 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:56.210448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:56.211143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.211423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.211698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.212485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] RBAC: Adding query filter params (role.id=3320a9786ec948aab8c8fcbb156eb9fa, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user.id=142d81749cb3486b87b11184e2cd8b6e) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:56.212794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:56.214296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:56.226949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 54/113] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:34:55 2022] GET /identity/v3/role_assignments?user.id=142d81749cb3486b87b11184e2cd8b6e&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=3320a9786ec948aab8c8fcbb156eb9fa => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:56.228267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:56.277987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:56.306200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999ce0a-abdc-4082-aecd-62231a9bbfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:56.306945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999ce0a-abdc-4082-aecd-62231a9bbfa1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.307234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999ce0a-abdc-4082-aecd-62231a9bbfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.307503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999ce0a-abdc-4082-aecd-62231a9bbfa1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.325723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999ce0a-abdc-4082-aecd-62231a9bbfa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-d999ce0a-abdc-4082-aecd-62231a9bbfa1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:56.330812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:56.335829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 60/114] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:34:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:56.336560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fff32-91bf-4b92-97d8-fb0887a47d34 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.337280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fff32-91bf-4b92-97d8-fb0887a47d34 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.337280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331fff32-91bf-4b92-97d8-fb0887a47d34 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.341199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 61/115] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:56.342100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fff32-91bf-4b92-97d8-fb0887a47d34 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:56.426069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22e47c-9ae0-46a0-8749-ab45d18c2067 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:56.469768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331fff32-91bf-4b92-97d8-fb0887a47d34 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:56.478841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba76378-d6b9-4f81-82e5-78bed47d0aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:56.479530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba76378-d6b9-4f81-82e5-78bed47d0aca None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.479816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba76378-d6b9-4f81-82e5-78bed47d0aca None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.480119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba76378-d6b9-4f81-82e5-78bed47d0aca None admin] PATH_INFO: `/v3/roles/member` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.486558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:56.498298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba76378-d6b9-4f81-82e5-78bed47d0aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-fba76378-d6b9-4f81-82e5-78bed47d0aca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:56.505617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:56.506815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:34:56.507789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:34:56.510027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 55/116] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:56.515766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e4cc2-afaa-4cb1-b7f9-b0d031fe5c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:56.515766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e4cc2-afaa-4cb1-b7f9-b0d031fe5c45 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.515766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e4cc2-afaa-4cb1-b7f9-b0d031fe5c45 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.515766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e4cc2-afaa-4cb1-b7f9-b0d031fe5c45 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.516146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e4cc2-afaa-4cb1-b7f9-b0d031fe5c45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:56.516146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e4cc2-afaa-4cb1-b7f9-b0d031fe5c45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:56.517638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e4cc2-afaa-4cb1-b7f9-b0d031fe5c45 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:56.518449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab313258-0ae4-4358-8393-f492603dade1 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.518886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab313258-0ae4-4358-8393-f492603dade1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.519545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab313258-0ae4-4358-8393-f492603dade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.529655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 62/117] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:56.530375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8e1ae-2b7a-4ada-b0a4-b093e0680cfa None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.530844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8e1ae-2b7a-4ada-b0a4-b093e0680cfa None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.531236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8e1ae-2b7a-4ada-b0a4-b093e0680cfa None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.532633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 63/118] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:56.533469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8e1ae-2b7a-4ada-b0a4-b093e0680cfa None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:56.542226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ab313258-0ae4-4358-8393-f492603dade1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:56.544685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ab313258-0ae4-4358-8393-f492603dade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:56.654342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 56/119] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:56.655703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab313258-0ae4-4358-8393-f492603dade1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:56.679224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8e1ae-2b7a-4ada-b0a4-b093e0680cfa None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:56.700617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:56.744551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55cab6-9c6d-4f1e-a910-0592a6da2d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:56.746418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55cab6-9c6d-4f1e-a910-0592a6da2d72 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.748031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55cab6-9c6d-4f1e-a910-0592a6da2d72 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.749407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55cab6-9c6d-4f1e-a910-0592a6da2d72 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.750288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55cab6-9c6d-4f1e-a910-0592a6da2d72 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:56.752044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55cab6-9c6d-4f1e-a910-0592a6da2d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:56.753779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55cab6-9c6d-4f1e-a910-0592a6da2d72 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:56.772907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 64/120] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:56.775514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681635a-d3a0-42e2-b1f8-fd3f8069d812 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.776157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681635a-d3a0-42e2-b1f8-fd3f8069d812 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.778394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681635a-d3a0-42e2-b1f8-fd3f8069d812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.814210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-b681635a-d3a0-42e2-b1f8-fd3f8069d812 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:56.817602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b681635a-d3a0-42e2-b1f8-fd3f8069d812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:56.852827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab313258-0ae4-4358-8393-f492603dade1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:56.879193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:56.919476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0428-7faa-4d68-ab8a-4b9aaad2cc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:56.920135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0428-7faa-4d68-ab8a-4b9aaad2cc65 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:56.920414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0428-7faa-4d68-ab8a-4b9aaad2cc65 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:56.920691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0428-7faa-4d68-ab8a-4b9aaad2cc65 None admin] PATH_INFO: `/v3/users/5f7a209666c0478690d711f7d361c03b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:56.924327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e0428-7faa-4d68-ab8a-4b9aaad2cc65 None admin] RBAC: Authorizing `identity:get_user(user_id=5f7a209666c0478690d711f7d361c03b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:56.925805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e0428-7faa-4d68-ab8a-4b9aaad2cc65 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:56.932343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 57/121] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:56 2022] GET /identity/v3/users/5f7a209666c0478690d711f7d361c03b => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:56.933248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0428-7faa-4d68-ab8a-4b9aaad2cc65 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:56.961613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 65/122] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:56.962493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681635a-d3a0-42e2-b1f8-fd3f8069d812 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.088730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681635a-d3a0-42e2-b1f8-fd3f8069d812 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.107278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.126066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256a597-7192-4ef2-b6f4-1cd6d9aa206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.126770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a597-7192-4ef2-b6f4-1cd6d9aa206b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.127087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a597-7192-4ef2-b6f4-1cd6d9aa206b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.127357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a597-7192-4ef2-b6f4-1cd6d9aa206b None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.128506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256a597-7192-4ef2-b6f4-1cd6d9aa206b None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:57.129948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256a597-7192-4ef2-b6f4-1cd6d9aa206b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:57.131626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 66/123] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:56 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:57.132389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256a597-7192-4ef2-b6f4-1cd6d9aa206b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.141642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0428-7faa-4d68-ab8a-4b9aaad2cc65 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.191515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.194644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ca357-8314-4527-844c-df12dd9d6fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.198785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ca357-8314-4527-844c-df12dd9d6fe9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.200184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ca357-8314-4527-844c-df12dd9d6fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.200438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ca357-8314-4527-844c-df12dd9d6fe9 None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.203777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ca357-8314-4527-844c-df12dd9d6fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:57.205313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ca357-8314-4527-844c-df12dd9d6fe9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:57.210501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 58/124] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:56 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:57.211319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ca357-8314-4527-844c-df12dd9d6fe9 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.325619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256a597-7192-4ef2-b6f4-1cd6d9aa206b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.381298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceaffd-b837-453b-b734-dc4dd085c8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.382066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceaffd-b837-453b-b734-dc4dd085c8db None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.382438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceaffd-b837-453b-b734-dc4dd085c8db None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.382726 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceaffd-b837-453b-b734-dc4dd085c8db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.392668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ca357-8314-4527-844c-df12dd9d6fe9 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.395791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbceaffd-b837-453b-b734-dc4dd085c8db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-fbceaffd-b837-453b-b734-dc4dd085c8db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:57.400793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:57.411922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 67/125] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:57.412877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceaffd-b837-453b-b734-dc4dd085c8db None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.440129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc36f2-a4c3-48c2-b13e-0a7f5a3c0ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.441226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc36f2-a4c3-48c2-b13e-0a7f5a3c0ff4 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.441647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc36f2-a4c3-48c2-b13e-0a7f5a3c0ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.442063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc36f2-a4c3-48c2-b13e-0a7f5a3c0ff4 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/users/5f7a209666c0478690d711f7d361c03b/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.444903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc36f2-a4c3-48c2-b13e-0a7f5a3c0ff4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user_id=5f7a209666c0478690d711f7d361c03b, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:57.447074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc36f2-a4c3-48c2-b13e-0a7f5a3c0ff4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:57.516770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.524433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 59/126] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:34:57 2022] PUT /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/users/5f7a209666c0478690d711f7d361c03b/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:34:57.524562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceaffd-b837-453b-b734-dc4dd085c8db None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.536162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc36f2-a4c3-48c2-b13e-0a7f5a3c0ff4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.562313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.575735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3fe88-dab6-4681-8394-b5ba1c4389e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.582811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3fe88-dab6-4681-8394-b5ba1c4389e9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.582811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3fe88-dab6-4681-8394-b5ba1c4389e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.582811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3fe88-dab6-4681-8394-b5ba1c4389e9 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.582811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3fe88-dab6-4681-8394-b5ba1c4389e9 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:57.582811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3fe88-dab6-4681-8394-b5ba1c4389e9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:57.584160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 68/127] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:57.586508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc62b4-801a-4d10-88d3-9bf1a32fb53d None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.587060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc62b4-801a-4d10-88d3-9bf1a32fb53d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.587541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc62b4-801a-4d10-88d3-9bf1a32fb53d None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.594448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 69/128] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:57.596088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc62b4-801a-4d10-88d3-9bf1a32fb53d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.672471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc36f2-a4c3-48c2-b13e-0a7f5a3c0ff4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.716488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb06ab8-e72b-43a7-bd0d-584d76ea150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.717476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06ab8-e72b-43a7-bd0d-584d76ea150f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.718000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06ab8-e72b-43a7-bd0d-584d76ea150f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.718493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06ab8-e72b-43a7-bd0d-584d76ea150f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.732683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb06ab8-e72b-43a7-bd0d-584d76ea150f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-3cb06ab8-e72b-43a7-bd0d-584d76ea150f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:57.738016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:57.738859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:34:57.739899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:34:57.741149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 60/129] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:57.741767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb06ab8-e72b-43a7-bd0d-584d76ea150f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:57.743908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc62b4-801a-4d10-88d3-9bf1a32fb53d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.806222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cbfc4-c55e-406d-87ad-d3fe1c95f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.806840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbfc4-c55e-406d-87ad-d3fe1c95f5dc None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.807144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbfc4-c55e-406d-87ad-d3fe1c95f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.807414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbfc4-c55e-406d-87ad-d3fe1c95f5dc None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.808136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cbfc4-c55e-406d-87ad-d3fe1c95f5dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:57.808481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cbfc4-c55e-406d-87ad-d3fe1c95f5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:57.809974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cbfc4-c55e-406d-87ad-d3fe1c95f5dc None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:57.822216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 70/130] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:57.823883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ba2e-b9f1-4d20-9cfe-febdf9e5f886 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.824545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ba2e-b9f1-4d20-9cfe-febdf9e5f886 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.825044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ba2e-b9f1-4d20-9cfe-febdf9e5f886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.844304 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-95d3ba2e-b9f1-4d20-9cfe-febdf9e5f886 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:57.845831 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-95d3ba2e-b9f1-4d20-9cfe-febdf9e5f886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:57.846077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.866535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:57.873488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb06ab8-e72b-43a7-bd0d-584d76ea150f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:57.913857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2960bc1-6d07-4e07-8f01-4e7994f62a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:57.914540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960bc1-6d07-4e07-8f01-4e7994f62a35 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:57.914817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960bc1-6d07-4e07-8f01-4e7994f62a35 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:57.915102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960bc1-6d07-4e07-8f01-4e7994f62a35 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/users/c7bb81025bd540828ef536c262646131/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:57.917134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2960bc1-6d07-4e07-8f01-4e7994f62a35 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user_id=c7bb81025bd540828ef536c262646131, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:57.919286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2960bc1-6d07-4e07-8f01-4e7994f62a35 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:57.927915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 71/131] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:57.929156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3ba2e-b9f1-4d20-9cfe-febdf9e5f886 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.009982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 61/132] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:34:57 2022] PUT /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/users/c7bb81025bd540828ef536c262646131/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:34:58.010814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2960bc1-6d07-4e07-8f01-4e7994f62a35 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.044270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3ba2e-b9f1-4d20-9cfe-febdf9e5f886 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.084472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.085456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.085948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.086443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.087369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:58.087895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.089647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.099381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 72/133] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:34:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:58.105384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.151314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.227796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2960bc1-6d07-4e07-8f01-4e7994f62a35 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.297995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e5bba-dc13-4ebf-82eb-813fb10654f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.299854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e5bba-dc13-4ebf-82eb-813fb10654f4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.300136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e5bba-dc13-4ebf-82eb-813fb10654f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.300409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e5bba-dc13-4ebf-82eb-813fb10654f4 None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.303826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e5bba-dc13-4ebf-82eb-813fb10654f4 None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.306823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e5bba-dc13-4ebf-82eb-813fb10654f4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.309124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 62/134] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:58.312794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e5bba-dc13-4ebf-82eb-813fb10654f4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.317670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.325322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34b1df-2896-4ddb-8462-82582dc378c9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.369676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10eab5-86e6-4b7c-a6de-f2c802d095b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.370669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10eab5-86e6-4b7c-a6de-f2c802d095b6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.371452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10eab5-86e6-4b7c-a6de-f2c802d095b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.372146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10eab5-86e6-4b7c-a6de-f2c802d095b6 None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.373658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10eab5-86e6-4b7c-a6de-f2c802d095b6 None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.376322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10eab5-86e6-4b7c-a6de-f2c802d095b6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.378171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 73/135] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:58.379154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10eab5-86e6-4b7c-a6de-f2c802d095b6 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.435101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e5bba-dc13-4ebf-82eb-813fb10654f4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.479662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e3ddc-e5e5-4042-9314-d8a8d80db3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.480285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3ddc-e5e5-4042-9314-d8a8d80db3d7 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.480579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3ddc-e5e5-4042-9314-d8a8d80db3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.481022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3ddc-e5e5-4042-9314-d8a8d80db3d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.482273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e3ddc-e5e5-4042-9314-d8a8d80db3d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.483960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e3ddc-e5e5-4042-9314-d8a8d80db3d7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.485576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 63/136] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:34:58.486544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.488175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c9997-e2ca-42e9-9ff5-687bbb26c03f None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.488661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c9997-e2ca-42e9-9ff5-687bbb26c03f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.489145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c9997-e2ca-42e9-9ff5-687bbb26c03f None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.491174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 64/137] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:34:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:34:58.496597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c9997-e2ca-42e9-9ff5-687bbb26c03f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.512428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10eab5-86e6-4b7c-a6de-f2c802d095b6 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.552354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc74a2-413f-4aa7-9295-828fe1a80ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.553394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc74a2-413f-4aa7-9295-828fe1a80ea1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.553860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc74a2-413f-4aa7-9295-828fe1a80ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.554315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc74a2-413f-4aa7-9295-828fe1a80ea1 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.555872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc74a2-413f-4aa7-9295-828fe1a80ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.557584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc74a2-413f-4aa7-9295-828fe1a80ea1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.560323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 74/138] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:58.561284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc74a2-413f-4aa7-9295-828fe1a80ea1 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.593333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.611351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c9997-e2ca-42e9-9ff5-687bbb26c03f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.650191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a34cc-7f9d-45d4-9598-a833c113d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.650833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a34cc-7f9d-45d4-9598-a833c113d227 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.651154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a34cc-7f9d-45d4-9598-a833c113d227 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.651427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a34cc-7f9d-45d4-9598-a833c113d227 None admin] PATH_INFO: `/v3/projects/cf5a7cedc7e34dafa19c08c291efae39` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.653143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a34cc-7f9d-45d4-9598-a833c113d227 None admin] RBAC: Authorizing `identity:get_project(project_id=cf5a7cedc7e34dafa19c08c291efae39)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.654847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a34cc-7f9d-45d4-9598-a833c113d227 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.657266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 65/139] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:58 2022] GET /identity/v3/projects/cf5a7cedc7e34dafa19c08c291efae39 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:58.658573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49d61b-5628-49ae-ab3b-3fbef358d661 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.658950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49d61b-5628-49ae-ab3b-3fbef358d661 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.659297 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49d61b-5628-49ae-ab3b-3fbef358d661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.679191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8b49d61b-5628-49ae-ab3b-3fbef358d661 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:34:58.680491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8b49d61b-5628-49ae-ab3b-3fbef358d661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:34:58.680828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc74a2-413f-4aa7-9295-828fe1a80ea1 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.718680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83429f6-b189-4adc-845d-43fb189d59ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.719356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83429f6-b189-4adc-845d-43fb189d59ef None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.719637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83429f6-b189-4adc-845d-43fb189d59ef None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.719912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83429f6-b189-4adc-845d-43fb189d59ef None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.720568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83429f6-b189-4adc-845d-43fb189d59ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.722019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83429f6-b189-4adc-845d-43fb189d59ef None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:58.759557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 66/140] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:34:58.760201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49d61b-5628-49ae-ab3b-3fbef358d661 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.806161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 75/141] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:34:58 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:34:58.807262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83429f6-b189-4adc-845d-43fb189d59ef None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:58.829531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.866838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:58.921370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49d61b-5628-49ae-ab3b-3fbef358d661 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:58.975053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebe14-a5ea-4f24-8d71-806c6bcfd4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:58.975829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebe14-a5ea-4f24-8d71-806c6bcfd4f1 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:58.976112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebe14-a5ea-4f24-8d71-806c6bcfd4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:58.976386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebe14-a5ea-4f24-8d71-806c6bcfd4f1 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/users/c7bb81025bd540828ef536c262646131/roles/98aaf5af7bb24b1596a9b01fd76eaf2c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:58.978708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ebe14-a5ea-4f24-8d71-806c6bcfd4f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user_id=c7bb81025bd540828ef536c262646131, role_id=98aaf5af7bb24b1596a9b01fd76eaf2c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:58.980982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ebe14-a5ea-4f24-8d71-806c6bcfd4f1 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.045828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 67/142] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:34:58 2022] PUT /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/users/c7bb81025bd540828ef536c262646131/roles/98aaf5af7bb24b1596a9b01fd76eaf2c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:34:59.082574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83429f6-b189-4adc-845d-43fb189d59ef None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.100038 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebe14-a5ea-4f24-8d71-806c6bcfd4f1 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:59.128483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfa929-f773-4c71-be9b-1c29e6543c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.129557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfa929-f773-4c71-be9b-1c29e6543c99 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.130124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfa929-f773-4c71-be9b-1c29e6543c99 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.130601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfa929-f773-4c71-be9b-1c29e6543c99 None admin] PATH_INFO: `/v3/projects/cf5a7cedc7e34dafa19c08c291efae39/users/c7bb81025bd540828ef536c262646131/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.132890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfa929-f773-4c71-be9b-1c29e6543c99 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf5a7cedc7e34dafa19c08c291efae39, user_id=c7bb81025bd540828ef536c262646131, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:59.134654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfa929-f773-4c71-be9b-1c29e6543c99 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.196623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 76/143] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:34:58 2022] PUT /identity/v3/projects/cf5a7cedc7e34dafa19c08c291efae39/users/c7bb81025bd540828ef536c262646131/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:34:59.253351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebe14-a5ea-4f24-8d71-806c6bcfd4f1 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.288643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:59.296138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ae851-8e4d-40bd-b2b9-8860aa8834f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.296715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ae851-8e4d-40bd-b2b9-8860aa8834f8 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.296992 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ae851-8e4d-40bd-b2b9-8860aa8834f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.297263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ae851-8e4d-40bd-b2b9-8860aa8834f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.310196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ae851-8e4d-40bd-b2b9-8860aa8834f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-428ae851-8e4d-40bd-b2b9-8860aa8834f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:34:59.315213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:34:59.316105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:34:59.316844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 68/144] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:34:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:34:59.323792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfa929-f773-4c71-be9b-1c29e6543c99 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:59.410989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:59.450589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfa929-f773-4c71-be9b-1c29e6543c99 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.489423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.490044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.490340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.490698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.491429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:34:59.491917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:59.493955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.504836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 77/145] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:34:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:59.510502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ae851-8e4d-40bd-b2b9-8860aa8834f8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:59.625038 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ae851-8e4d-40bd-b2b9-8860aa8834f8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.662270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f13b65-a8ab-45c5-be3a-9032f8a45692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.662995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f13b65-a8ab-45c5-be3a-9032f8a45692 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.663270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f13b65-a8ab-45c5-be3a-9032f8a45692 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.663545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f13b65-a8ab-45c5-be3a-9032f8a45692 None admin] PATH_INFO: `/v3/users/142d81749cb3486b87b11184e2cd8b6e` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.664724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f13b65-a8ab-45c5-be3a-9032f8a45692 None admin] RBAC: Authorizing `identity:get_user(user_id=142d81749cb3486b87b11184e2cd8b6e)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:59.666248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f13b65-a8ab-45c5-be3a-9032f8a45692 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.670314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 69/146] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:34:59 2022] GET /identity/v3/users/142d81749cb3486b87b11184e2cd8b6e => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:59.676027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:59.791014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292265bb-9aef-4ab7-9db2-03663413ded7 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:34:59.823146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:59.829860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4839aed-500e-4978-bd30-53d2eac94c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:34:59.830439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839aed-500e-4978-bd30-53d2eac94c6a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:34:59.830716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839aed-500e-4978-bd30-53d2eac94c6a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:34:59.831003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839aed-500e-4978-bd30-53d2eac94c6a None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:34:59.832165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4839aed-500e-4978-bd30-53d2eac94c6a None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:34:59.833627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4839aed-500e-4978-bd30-53d2eac94c6a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:34:59.835384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 78/147] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:34:59 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:34:59.841408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f13b65-a8ab-45c5-be3a-9032f8a45692 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:34:59.917149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:34:59.962962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f13b65-a8ab-45c5-be3a-9032f8a45692 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:00.002118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5dd18-fc52-47d2-b9dc-ee5ddfda9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:00.002799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5dd18-fc52-47d2-b9dc-ee5ddfda9671 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:00.003115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5dd18-fc52-47d2-b9dc-ee5ddfda9671 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:00.003386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5dd18-fc52-47d2-b9dc-ee5ddfda9671 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/users/142d81749cb3486b87b11184e2cd8b6e/roles/3320a9786ec948aab8c8fcbb156eb9fa` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:00.005371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5dd18-fc52-47d2-b9dc-ee5ddfda9671 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user_id=142d81749cb3486b87b11184e2cd8b6e, role_id=3320a9786ec948aab8c8fcbb156eb9fa)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:00.006819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5dd18-fc52-47d2-b9dc-ee5ddfda9671 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:00.081002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 70/148] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:34:59 2022] PUT /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/users/142d81749cb3486b87b11184e2cd8b6e/roles/3320a9786ec948aab8c8fcbb156eb9fa => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:35:00.125715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e497-b798-48ec-a73d-7111f37cbb8c None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:00.126149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e497-b798-48ec-a73d-7111f37cbb8c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:00.126549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e497-b798-48ec-a73d-7111f37cbb8c None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:00.128410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 79/149] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:00.135025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04afa7-3ae0-4f1b-867a-511b058a18b9 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:00.135356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04afa7-3ae0-4f1b-867a-511b058a18b9 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:00.135644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04afa7-3ae0-4f1b-867a-511b058a18b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:00.161406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7f04afa7-3ae0-4f1b-867a-511b058a18b9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:00.162732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7f04afa7-3ae0-4f1b-867a-511b058a18b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:00.259797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 71/150] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:00.709153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0e497-b798-48ec-a73d-7111f37cbb8c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:00.842129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0e497-b798-48ec-a73d-7111f37cbb8c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:00.883728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6419de-e469-4ee3-8924-e0e4c293b8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:00.884382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6419de-e469-4ee3-8924-e0e4c293b8bc None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:00.884658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6419de-e469-4ee3-8924-e0e4c293b8bc None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:00.884930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6419de-e469-4ee3-8924-e0e4c293b8bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:00.897569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6419de-e469-4ee3-8924-e0e4c293b8bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-be6419de-e469-4ee3-8924-e0e4c293b8bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:00.903019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:00.905501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:00.907432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 80/151] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:00.914956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04afa7-3ae0-4f1b-867a-511b058a18b9 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:00.966133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:01.026512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04afa7-3ae0-4f1b-867a-511b058a18b9 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:01.075674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:01.076306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.076585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.076869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.077553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:01.077852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:01.079273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:01.088318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 72/152] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:01.093944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6419de-e469-4ee3-8924-e0e4c293b8bc None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:01.171689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:01.217257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6419de-e469-4ee3-8924-e0e4c293b8bc None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:01.258361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00085561-e0ff-4c0c-bc34-b7e00021377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:01.259025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00085561-e0ff-4c0c-bc34-b7e00021377a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.259354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00085561-e0ff-4c0c-bc34-b7e00021377a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.259629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00085561-e0ff-4c0c-bc34-b7e00021377a None admin] PATH_INFO: `/v3/users/5f7a209666c0478690d711f7d361c03b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.260771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00085561-e0ff-4c0c-bc34-b7e00021377a None admin] RBAC: Authorizing `identity:get_user(user_id=5f7a209666c0478690d711f7d361c03b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:01.262371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00085561-e0ff-4c0c-bc34-b7e00021377a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:01.264010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 81/153] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:01 2022] GET /identity/v3/users/5f7a209666c0478690d711f7d361c03b => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:01.269661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:01.409049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5955b61-7eb6-472d-8cca-d90521ec1234 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:01.420357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:01.428801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca27b4-8178-445f-a2f3-a834dc60eabe None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.434774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca27b4-8178-445f-a2f3-a834dc60eabe None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.434774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca27b4-8178-445f-a2f3-a834dc60eabe None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.434774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 82/154] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:01.445108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58424f-2e56-42c6-94ac-bf7d18d32daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:01.445728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58424f-2e56-42c6-94ac-bf7d18d32daf None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.446000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58424f-2e56-42c6-94ac-bf7d18d32daf None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.446329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58424f-2e56-42c6-94ac-bf7d18d32daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.452396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ab8d8-f10c-4ad2-93ed-594677a8744c None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.452675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ab8d8-f10c-4ad2-93ed-594677a8744c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.452945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ab8d8-f10c-4ad2-93ed-594677a8744c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.493538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ad8ab8d8-f10c-4ad2-93ed-594677a8744c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:01.494662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ad8ab8d8-f10c-4ad2-93ed-594677a8744c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:01.521371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58424f-2e56-42c6-94ac-bf7d18d32daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:01.522862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58424f-2e56-42c6-94ac-bf7d18d32daf None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:01.621325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 83/155] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:01.632113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 73/156] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:01.637900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ab8d8-f10c-4ad2-93ed-594677a8744c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:01.727400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:01.747126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ab8d8-f10c-4ad2-93ed-594677a8744c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:01.783366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c959d-4b93-48e4-bef9-afd59df7b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:01.784286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c959d-4b93-48e4-bef9-afd59df7b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.784728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c959d-4b93-48e4-bef9-afd59df7b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.785189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c959d-4b93-48e4-bef9-afd59df7b4c9 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.786491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c959d-4b93-48e4-bef9-afd59df7b4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:01.788137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c959d-4b93-48e4-bef9-afd59df7b4c9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:01.790295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 84/157] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:01 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:01.797048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58424f-2e56-42c6-94ac-bf7d18d32daf None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:01.809994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:01.904757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58424f-2e56-42c6-94ac-bf7d18d32daf None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:01.942124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:01.943119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:01.943578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:01.944016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:01.944923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user.id=5f7a209666c0478690d711f7d361c03b) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:01.945399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:01.947042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:01.954634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 74/158] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:01 2022] GET /identity/v3/role_assignments?user.id=5f7a209666c0478690d711f7d361c03b&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:02.316062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c959d-4b93-48e4-bef9-afd59df7b4c9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.442590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c959d-4b93-48e4-bef9-afd59df7b4c9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.469346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:02.482728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01788d4-cc1c-4053-ae5d-fc8ad1a5d069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.483410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01788d4-cc1c-4053-ae5d-fc8ad1a5d069 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.489546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01788d4-cc1c-4053-ae5d-fc8ad1a5d069 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.489546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01788d4-cc1c-4053-ae5d-fc8ad1a5d069 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.495463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01788d4-cc1c-4053-ae5d-fc8ad1a5d069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e01788d4-cc1c-4053-ae5d-fc8ad1a5d069 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:02.500447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:02.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:02.502358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 85/159] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:02.509015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.594135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:02.621214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973df43c-2bc0-46dc-939b-d9d3c45d8b51 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.657115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47576a2b-1934-45ca-b66d-af26e3cc1e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.657733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47576a2b-1934-45ca-b66d-af26e3cc1e5b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.658008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47576a2b-1934-45ca-b66d-af26e3cc1e5b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.658271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47576a2b-1934-45ca-b66d-af26e3cc1e5b None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.659013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47576a2b-1934-45ca-b66d-af26e3cc1e5b None admin] RBAC: Adding query filter params (name=member) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:02.659310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47576a2b-1934-45ca-b66d-af26e3cc1e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:02.660681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47576a2b-1934-45ca-b66d-af26e3cc1e5b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:02.670461 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 75/160] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:02.675692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01788d4-cc1c-4053-ae5d-fc8ad1a5d069 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.809377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01788d4-cc1c-4053-ae5d-fc8ad1a5d069 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:02.838442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed32311-7b52-4d01-9152-60006aaf06fb None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.838737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed32311-7b52-4d01-9152-60006aaf06fb None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.838905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed32311-7b52-4d01-9152-60006aaf06fb None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.840498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 76/161] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:02.847714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed205958-7713-4839-a69c-6dfbffee5a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:02.848362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed205958-7713-4839-a69c-6dfbffee5a0b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.848637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed205958-7713-4839-a69c-6dfbffee5a0b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.848907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed205958-7713-4839-a69c-6dfbffee5a0b None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.851488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed205958-7713-4839-a69c-6dfbffee5a0b None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:02.852389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3720fe-4e18-4780-99a5-aff12b849ff0 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:02.852676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3720fe-4e18-4780-99a5-aff12b849ff0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:02.852945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed205958-7713-4839-a69c-6dfbffee5a0b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:02.853052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3720fe-4e18-4780-99a5-aff12b849ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:02.854766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 86/162] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:02.860722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed205958-7713-4839-a69c-6dfbffee5a0b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:02.888537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8e3720fe-4e18-4780-99a5-aff12b849ff0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:02.889754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8e3720fe-4e18-4780-99a5-aff12b849ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:02.943434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:02.998781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed205958-7713-4839-a69c-6dfbffee5a0b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:03.040579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ae20b-1009-41b9-974c-677eebe24cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:03.041682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae20b-1009-41b9-974c-677eebe24cad None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.042096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae20b-1009-41b9-974c-677eebe24cad None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.043279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae20b-1009-41b9-974c-677eebe24cad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.059783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 77/163] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:03.125727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ae20b-1009-41b9-974c-677eebe24cad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:03.127252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ae20b-1009-41b9-974c-677eebe24cad None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:03.131937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebcffd-4584-406b-a93f-292e3d452681 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.132226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebcffd-4584-406b-a93f-292e3d452681 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.132498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebcffd-4584-406b-a93f-292e3d452681 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.133983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 78/164] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:03.151593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43eb0ee-7fc3-488a-b203-2c2159a9e3dc None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.152475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43eb0ee-7fc3-488a-b203-2c2159a9e3dc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.152970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43eb0ee-7fc3-488a-b203-2c2159a9e3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.183445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:03.200542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-c43eb0ee-7fc3-488a-b203-2c2159a9e3dc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:03.201760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c43eb0ee-7fc3-488a-b203-2c2159a9e3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:03.236494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 87/165] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:35:03.242302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ae20b-1009-41b9-974c-677eebe24cad None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:03.352848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ae20b-1009-41b9-974c-677eebe24cad None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:03.359993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:03.384842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 79/166] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:03.384842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e9b72-2258-4e63-8e8e-6c49da19ca8f None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.384842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e9b72-2258-4e63-8e8e-6c49da19ca8f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.384842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e9b72-2258-4e63-8e8e-6c49da19ca8f None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.384842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 80/167] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:03.396285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc151c57-9646-4d97-bb29-69d5c467268c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.396962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc151c57-9646-4d97-bb29-69d5c467268c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.397383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c9754-b858-4140-ad3e-6f68d6a8e953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:03.397531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc151c57-9646-4d97-bb29-69d5c467268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.398356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c9754-b858-4140-ad3e-6f68d6a8e953 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.398823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c9754-b858-4140-ad3e-6f68d6a8e953 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.399296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c9754-b858-4140-ad3e-6f68d6a8e953 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.403699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c9754-b858-4140-ad3e-6f68d6a8e953 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:03.407609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c9754-b858-4140-ad3e-6f68d6a8e953 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:03.410649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 88/168] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:03 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:03.416839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c9754-b858-4140-ad3e-6f68d6a8e953 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:03.438268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-fc151c57-9646-4d97-bb29-69d5c467268c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:03.441259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-fc151c57-9646-4d97-bb29-69d5c467268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:03.465395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:03.596552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 81/169] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:03.597466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc151c57-9646-4d97-bb29-69d5c467268c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:03.684675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c9754-b858-4140-ad3e-6f68d6a8e953 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:03.754241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:03.755005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.755253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.755521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.756258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] RBAC: Adding query filter params (role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user.id=c7bb81025bd540828ef536c262646131) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:03.756557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:03.758018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:03.770601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 89/170] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:03 2022] GET /identity/v3/role_assignments?user.id=c7bb81025bd540828ef536c262646131&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 608 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:03.809839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc151c57-9646-4d97-bb29-69d5c467268c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:03.849040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:03.858892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aee955-bc0d-4e65-b5c1-59a58209a508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:03.860641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aee955-bc0d-4e65-b5c1-59a58209a508 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:03.860641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aee955-bc0d-4e65-b5c1-59a58209a508 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:03.860641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aee955-bc0d-4e65-b5c1-59a58209a508 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:03.873924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aee955-bc0d-4e65-b5c1-59a58209a508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-99aee955-bc0d-4e65-b5c1-59a58209a508 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:03.878729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:03.879613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:03.880389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 82/171] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:03.885926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:03.888267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aee955-bc0d-4e65-b5c1-59a58209a508 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.000063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:04.016431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c17e9-3cb7-493a-b2da-6fb5ea362ce0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.094740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.095392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.095672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.095947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.096623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:04.096924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:04.098308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:04.105712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aee955-bc0d-4e65-b5c1-59a58209a508 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.107090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 90/172] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:04.112843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.156280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc0cab-5e2e-42c4-831c-19cdb7890f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.156918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc0cab-5e2e-42c4-831c-19cdb7890f5f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.157189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc0cab-5e2e-42c4-831c-19cdb7890f5f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.157452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc0cab-5e2e-42c4-831c-19cdb7890f5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.172116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc0cab-5e2e-42c4-831c-19cdb7890f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-93fc0cab-5e2e-42c4-831c-19cdb7890f5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:04.177307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:04.179318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:04.181168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 83/173] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:04.186948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc0cab-5e2e-42c4-831c-19cdb7890f5f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.243950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:04.260910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2ec94-2f74-4599-b32d-e81f19be632f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.303276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc0cab-5e2e-42c4-831c-19cdb7890f5f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.304270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94308902-fb0c-4374-88f2-8faa0da0075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.305100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94308902-fb0c-4374-88f2-8faa0da0075b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.305379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94308902-fb0c-4374-88f2-8faa0da0075b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.306457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94308902-fb0c-4374-88f2-8faa0da0075b None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.306849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94308902-fb0c-4374-88f2-8faa0da0075b None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:04.309331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94308902-fb0c-4374-88f2-8faa0da0075b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:04.311225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 91/174] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:04 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:04.316985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1515e2-724f-47bc-bdb2-00e51682dab7 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.317271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1515e2-724f-47bc-bdb2-00e51682dab7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.317581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1515e2-724f-47bc-bdb2-00e51682dab7 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.319117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 92/175] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:04.319976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1515e2-724f-47bc-bdb2-00e51682dab7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.338419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:04.345708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a060bad-490b-4c0f-92aa-830f602e29fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.346325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a060bad-490b-4c0f-92aa-830f602e29fb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.346956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a060bad-490b-4c0f-92aa-830f602e29fb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.346956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a060bad-490b-4c0f-92aa-830f602e29fb None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.347625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a060bad-490b-4c0f-92aa-830f602e29fb None admin] RBAC: Adding query filter params (name=member) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:04.347930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a060bad-490b-4c0f-92aa-830f602e29fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:04.349323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a060bad-490b-4c0f-92aa-830f602e29fb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:04.359372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 84/176] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:04.360614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfb95-f22f-4ca5-a295-f10d380f2b40 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.360895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfb95-f22f-4ca5-a295-f10d380f2b40 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.361163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bfb95-f22f-4ca5-a295-f10d380f2b40 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.366538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 85/177] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:04.366538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bfb95-f22f-4ca5-a295-f10d380f2b40 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.447123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1515e2-724f-47bc-bdb2-00e51682dab7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.480538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bfb95-f22f-4ca5-a295-f10d380f2b40 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.486959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afe80c-3440-4af3-ac14-ad75fe3e763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.487608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afe80c-3440-4af3-ac14-ad75fe3e763e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.487883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afe80c-3440-4af3-ac14-ad75fe3e763e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.488154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afe80c-3440-4af3-ac14-ad75fe3e763e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.490980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:04.503808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afe80c-3440-4af3-ac14-ad75fe3e763e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-d7afe80c-3440-4af3-ac14-ad75fe3e763e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:04.509534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:04.510414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:04.512184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 93/178] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:04.513423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03be43-a6a3-4b85-a001-52934afb023c None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.513423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03be43-a6a3-4b85-a001-52934afb023c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.513423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03be43-a6a3-4b85-a001-52934afb023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.519641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159179b-0b8a-4930-9a03-de5376547a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.520258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159179b-0b8a-4930-9a03-de5376547a32 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.520536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159179b-0b8a-4930-9a03-de5376547a32 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.520807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159179b-0b8a-4930-9a03-de5376547a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.531729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7a03be43-a6a3-4b85-a001-52934afb023c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:04.532926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7a03be43-a6a3-4b85-a001-52934afb023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:04.593210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159179b-0b8a-4930-9a03-de5376547a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:04.594749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159179b-0b8a-4930-9a03-de5376547a32 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:04.599848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 94/179] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:04.600692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03be43-a6a3-4b85-a001-52934afb023c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.604559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:04.707821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03be43-a6a3-4b85-a001-52934afb023c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.713625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 86/180] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:04.714938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42597d5c-dd5e-44d9-aa70-e94c236da2b2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.715410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42597d5c-dd5e-44d9-aa70-e94c236da2b2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.715682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42597d5c-dd5e-44d9-aa70-e94c236da2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.733161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-42597d5c-dd5e-44d9-aa70-e94c236da2b2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:04.734346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-42597d5c-dd5e-44d9-aa70-e94c236da2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:04.736504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:04.750344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273c3d5-5a24-484d-a99d-0bd0f6e72df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.751025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273c3d5-5a24-484d-a99d-0bd0f6e72df2 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.751308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273c3d5-5a24-484d-a99d-0bd0f6e72df2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.751580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273c3d5-5a24-484d-a99d-0bd0f6e72df2 None admin] PATH_INFO: `/v3/users/c7bb81025bd540828ef536c262646131` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.752726 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273c3d5-5a24-484d-a99d-0bd0f6e72df2 None admin] RBAC: Authorizing `identity:get_user(user_id=c7bb81025bd540828ef536c262646131)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:04.754187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273c3d5-5a24-484d-a99d-0bd0f6e72df2 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:04.755840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 95/181] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:04 2022] GET /identity/v3/users/c7bb81025bd540828ef536c262646131 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:04.757034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf2a3-4300-4064-9941-47ea52a33c46 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.757354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf2a3-4300-4064-9941-47ea52a33c46 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.757643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf2a3-4300-4064-9941-47ea52a33c46 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.763024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 96/182] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:04.763024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf2a3-4300-4064-9941-47ea52a33c46 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.814013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 87/183] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:04.814674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42597d5c-dd5e-44d9-aa70-e94c236da2b2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.843907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:04.879539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf2a3-4300-4064-9941-47ea52a33c46 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.919223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.920762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.921240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.921705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.922202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42597d5c-dd5e-44d9-aa70-e94c236da2b2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:04.922832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:04.923302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:04.925043 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:04.934860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 97/184] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:04.935962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:04.962352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76447ec-69a6-4002-b45f-835fe01c4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:04.963086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76447ec-69a6-4002-b45f-835fe01c4c45 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.963363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76447ec-69a6-4002-b45f-835fe01c4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.963630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76447ec-69a6-4002-b45f-835fe01c4c45 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.965493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76447ec-69a6-4002-b45f-835fe01c4c45 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:04.967143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76447ec-69a6-4002-b45f-835fe01c4c45 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:04.973592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 88/185] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:04 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:04.973592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dff1a-e977-4137-9896-238035410ee3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:04.973592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dff1a-e977-4137-9896-238035410ee3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:04.973592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dff1a-e977-4137-9896-238035410ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:04.984610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:04.990975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b88dff1a-e977-4137-9896-238035410ee3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:04.992458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b88dff1a-e977-4137-9896-238035410ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:05.054407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ffc98-2fd7-44f5-83b0-d26358d9dae4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.068832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 89/186] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:05.070169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dff1a-e977-4137-9896-238035410ee3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.101400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cabb-822e-44ad-a1ef-6f45745bbaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.102012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cabb-822e-44ad-a1ef-6f45745bbaee None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.102289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cabb-822e-44ad-a1ef-6f45745bbaee None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.102563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734cabb-822e-44ad-a1ef-6f45745bbaee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.105769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:05.176286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cabb-822e-44ad-a1ef-6f45745bbaee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:05.177758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734cabb-822e-44ad-a1ef-6f45745bbaee None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:05.195650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dff1a-e977-4137-9896-238035410ee3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.235339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340fa3-daf4-4bc3-ae92-0eb3448f70e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.235984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340fa3-daf4-4bc3-ae92-0eb3448f70e4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.236382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340fa3-daf4-4bc3-ae92-0eb3448f70e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.236654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340fa3-daf4-4bc3-ae92-0eb3448f70e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.240081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:05.249197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90340fa3-daf4-4bc3-ae92-0eb3448f70e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-90340fa3-daf4-4bc3-ae92-0eb3448f70e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:05.253957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:05.254877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:05.255712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 90/187] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:05.256354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340fa3-daf4-4bc3-ae92-0eb3448f70e4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.292889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 98/188] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:05.298744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cabb-822e-44ad-a1ef-6f45745bbaee None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.336030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:05.368776 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340fa3-daf4-4bc3-ae92-0eb3448f70e4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.404395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b353e-87f5-4067-9d4f-b732d7f0e7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.405058 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b353e-87f5-4067-9d4f-b732d7f0e7a2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.405384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b353e-87f5-4067-9d4f-b732d7f0e7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.405650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b353e-87f5-4067-9d4f-b732d7f0e7a2 None admin] PATH_INFO: `/v3/users/59b49efadab246e1aec3f04209f16263` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.406048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734cabb-822e-44ad-a1ef-6f45745bbaee None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.422262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b353e-87f5-4067-9d4f-b732d7f0e7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=59b49efadab246e1aec3f04209f16263)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:05.423766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b353e-87f5-4067-9d4f-b732d7f0e7a2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:05.427569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 91/189] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/users/59b49efadab246e1aec3f04209f16263 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:05.427569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b353e-87f5-4067-9d4f-b732d7f0e7a2 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.442411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.443040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.443312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.443666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.444382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] RBAC: Adding query filter params (role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user.id=c7bb81025bd540828ef536c262646131) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:05.444680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:05.446122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:05.453553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 99/190] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/role_assignments?user.id=c7bb81025bd540828ef536c262646131&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:05.454169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.506893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:05.556920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b353e-87f5-4067-9d4f-b732d7f0e7a2 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.582780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:05.588328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70176c1d-c534-46cc-90db-1dcd4af943e1 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.603546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e95df6-1ea1-4d01-ba6e-4f5effd92d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.604205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e95df6-1ea1-4d01-ba6e-4f5effd92d49 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.604483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e95df6-1ea1-4d01-ba6e-4f5effd92d49 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.604782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e95df6-1ea1-4d01-ba6e-4f5effd92d49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.620335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e95df6-1ea1-4d01-ba6e-4f5effd92d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-f2e95df6-1ea1-4d01-ba6e-4f5effd92d49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:05.625913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:05.627011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 92/191] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:05.628999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e95df6-1ea1-4d01-ba6e-4f5effd92d49 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.644102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.645038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.645468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.645738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.646411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:05.646713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:05.648153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:05.659017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 100/192] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:05.660214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.743879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e95df6-1ea1-4d01-ba6e-4f5effd92d49 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.758739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:05.766580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9cf3c-0580-43b7-bf9a-2591613649ef None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.779885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c764ad-f253-44af-a07e-dcee25a4e17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.780511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c764ad-f253-44af-a07e-dcee25a4e17e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.780786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c764ad-f253-44af-a07e-dcee25a4e17e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.781050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c764ad-f253-44af-a07e-dcee25a4e17e None admin] PATH_INFO: `/v3/projects/cf5a7cedc7e34dafa19c08c291efae39` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.782163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c764ad-f253-44af-a07e-dcee25a4e17e None admin] RBAC: Authorizing `identity:get_project(project_id=cf5a7cedc7e34dafa19c08c291efae39)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:05.783649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c764ad-f253-44af-a07e-dcee25a4e17e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:05.785284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 93/193] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/projects/cf5a7cedc7e34dafa19c08c291efae39 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:05.786039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c764ad-f253-44af-a07e-dcee25a4e17e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.803033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888db618-3a33-4cea-8ac6-6430836a9916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.803674 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888db618-3a33-4cea-8ac6-6430836a9916 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.803948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888db618-3a33-4cea-8ac6-6430836a9916 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.804216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888db618-3a33-4cea-8ac6-6430836a9916 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.805414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888db618-3a33-4cea-8ac6-6430836a9916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:05.806954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888db618-3a33-4cea-8ac6-6430836a9916 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:05.809390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 101/194] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:05.811367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888db618-3a33-4cea-8ac6-6430836a9916 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.839730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:05.897721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c764ad-f253-44af-a07e-dcee25a4e17e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.913434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888db618-3a33-4cea-8ac6-6430836a9916 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:05.931944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e5f87-4e1d-4528-b046-d2c40282c2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.932555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e5f87-4e1d-4528-b046-d2c40282c2ab None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.932831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e5f87-4e1d-4528-b046-d2c40282c2ab None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.933154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e5f87-4e1d-4528-b046-d2c40282c2ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.947835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:05.948474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:05.948733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:05.949001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:05.949652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:05.949945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:05.951339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:05.959238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 102/195] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:05.960025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:05.984105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.005114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e5f87-4e1d-4528-b046-d2c40282c2ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.006831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e5f87-4e1d-4528-b046-d2c40282c2ab None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.053445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.064691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed298c6d-cebb-4c27-8dee-57ceb918c603 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.099337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6146e-006d-46a0-8819-e8843a3b1833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.099892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6146e-006d-46a0-8819-e8843a3b1833 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.100168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6146e-006d-46a0-8819-e8843a3b1833 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.100437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6146e-006d-46a0-8819-e8843a3b1833 None admin] PATH_INFO: `/v3/users/142d81749cb3486b87b11184e2cd8b6e` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.101562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6146e-006d-46a0-8819-e8843a3b1833 None admin] RBAC: Authorizing `identity:get_user(user_id=142d81749cb3486b87b11184e2cd8b6e)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.103040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6146e-006d-46a0-8819-e8843a3b1833 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.104624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 103/196] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/users/142d81749cb3486b87b11184e2cd8b6e => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:06.104624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6146e-006d-46a0-8819-e8843a3b1833 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.113990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 94/197] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:06.114821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e5f87-4e1d-4528-b046-d2c40282c2ab None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.195308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.206557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6146e-006d-46a0-8819-e8843a3b1833 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.213755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e5f87-4e1d-4528-b046-d2c40282c2ab None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.243410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.244023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.244301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.244741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.245315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] RBAC: Adding query filter params (role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=cf5a7cedc7e34dafa19c08c291efae39, user.id=c7bb81025bd540828ef536c262646131) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:06.246302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.248035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.250006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7668073-86b8-49f0-94ea-54acd41e9f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.250603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668073-86b8-49f0-94ea-54acd41e9f98 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.251072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668073-86b8-49f0-94ea-54acd41e9f98 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.251213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7668073-86b8-49f0-94ea-54acd41e9f98 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.251799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7668073-86b8-49f0-94ea-54acd41e9f98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.253199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7668073-86b8-49f0-94ea-54acd41e9f98 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.254173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 104/198] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/role_assignments?user.id=c7bb81025bd540828ef536c262646131&scope.project.id=cf5a7cedc7e34dafa19c08c291efae39&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:06.255050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.284429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.366339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5749a9a-a820-47a6-b2ed-0fe85e4ff675 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.394745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 95/199] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:06 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:06.404229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7668073-86b8-49f0-94ea-54acd41e9f98 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.406762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eea6d3-a3ff-4da2-950c-8a2f0265a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.407632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eea6d3-a3ff-4da2-950c-8a2f0265a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.407908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eea6d3-a3ff-4da2-950c-8a2f0265a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.408179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eea6d3-a3ff-4da2-950c-8a2f0265a8b5 None admin] PATH_INFO: `/v3/users/59b49efadab246e1aec3f04209f16263` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.409549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eea6d3-a3ff-4da2-950c-8a2f0265a8b5 None admin] RBAC: Authorizing `identity:get_user(user_id=59b49efadab246e1aec3f04209f16263)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.411125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eea6d3-a3ff-4da2-950c-8a2f0265a8b5 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.412860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 105/200] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/users/59b49efadab246e1aec3f04209f16263 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:06.413654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eea6d3-a3ff-4da2-950c-8a2f0265a8b5 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.547312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.560335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eea6d3-a3ff-4da2-950c-8a2f0265a8b5 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.565224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7668073-86b8-49f0-94ea-54acd41e9f98 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.603192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c8938-f858-4224-8ad4-45b5ab21a5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.603809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8938-f858-4224-8ad4-45b5ab21a5b1 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.604087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8938-f858-4224-8ad4-45b5ab21a5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.604357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8938-f858-4224-8ad4-45b5ab21a5b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.608066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c192e-7dd0-4461-86db-f3fec60b66c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.608955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c192e-7dd0-4461-86db-f3fec60b66c2 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.608955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c192e-7dd0-4461-86db-f3fec60b66c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.609138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c192e-7dd0-4461-86db-f3fec60b66c2 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.610968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c192e-7dd0-4461-86db-f3fec60b66c2 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.611730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c192e-7dd0-4461-86db-f3fec60b66c2 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.613538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 106/201] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:06.614951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c192e-7dd0-4461-86db-f3fec60b66c2 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.616685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.690607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c8938-f858-4224-8ad4-45b5ab21a5b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.692336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c8938-f858-4224-8ad4-45b5ab21a5b1 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.744346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c192e-7dd0-4461-86db-f3fec60b66c2 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:06.791064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fbba0-8a0a-400a-8f4c-878b61c302d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:06.791786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fbba0-8a0a-400a-8f4c-878b61c302d4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:06.792066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fbba0-8a0a-400a-8f4c-878b61c302d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:06.792336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fbba0-8a0a-400a-8f4c-878b61c302d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:06.820606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 96/202] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:06.822266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c8938-f858-4224-8ad4-45b5ab21a5b1 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:06.832388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.881913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fbba0-8a0a-400a-8f4c-878b61c302d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:06.883331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fbba0-8a0a-400a-8f4c-878b61c302d4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:06.885454 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:06.974467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c8938-f858-4224-8ad4-45b5ab21a5b1 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.001590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 107/203] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:07.002635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fbba0-8a0a-400a-8f4c-878b61c302d4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.022024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.023062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.023341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.023609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.024341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=f618dc420f704f2c8a85cfe182a07300, user.id=59b49efadab246e1aec3f04209f16263) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:07.024644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.026647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.034990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 97/204] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:06 2022] GET /identity/v3/role_assignments?user.id=59b49efadab246e1aec3f04209f16263&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:07.036112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.122956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:07.170876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370ba78-c958-4bb2-bfbf-8433649da278 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.172569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:07.225253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea279b4-cd62-43a4-8164-f2654a33ee71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.225908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea279b4-cd62-43a4-8164-f2654a33ee71 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.226190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea279b4-cd62-43a4-8164-f2654a33ee71 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.226458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea279b4-cd62-43a4-8164-f2654a33ee71 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.229458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fbba0-8a0a-400a-8f4c-878b61c302d4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.232117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea279b4-cd62-43a4-8164-f2654a33ee71 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.233590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea279b4-cd62-43a4-8164-f2654a33ee71 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.235462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 98/205] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:07.244463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea279b4-cd62-43a4-8164-f2654a33ee71 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.303016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a91e3f-9311-4ce7-bc93-a05dd53e3839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.303743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a91e3f-9311-4ce7-bc93-a05dd53e3839 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.304021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a91e3f-9311-4ce7-bc93-a05dd53e3839 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.304289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a91e3f-9311-4ce7-bc93-a05dd53e3839 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.305454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a91e3f-9311-4ce7-bc93-a05dd53e3839 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.306932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a91e3f-9311-4ce7-bc93-a05dd53e3839 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.313300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 108/206] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:07.315098 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a91e3f-9311-4ce7-bc93-a05dd53e3839 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:07.406227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea279b4-cd62-43a4-8164-f2654a33ee71 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.443302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f40de-4063-4652-b3d9-e6f0357928fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.443836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f40de-4063-4652-b3d9-e6f0357928fd None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.444213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f40de-4063-4652-b3d9-e6f0357928fd None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.444546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f40de-4063-4652-b3d9-e6f0357928fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.445394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f40de-4063-4652-b3d9-e6f0357928fd None admin] RBAC: Adding query filter params (role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=f618dc420f704f2c8a85cfe182a07300, user.id=59b49efadab246e1aec3f04209f16263) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:07.445743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f40de-4063-4652-b3d9-e6f0357928fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.447233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f40de-4063-4652-b3d9-e6f0357928fd None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.454541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 99/207] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/role_assignments?user.id=59b49efadab246e1aec3f04209f16263&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:07.461547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a91e3f-9311-4ce7-bc93-a05dd53e3839 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:07.510596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:07.519901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a253b-755d-4261-a5e7-542f174e4772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:07.520531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a253b-755d-4261-a5e7-542f174e4772 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:07.520806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a253b-755d-4261-a5e7-542f174e4772 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:07.521077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a253b-755d-4261-a5e7-542f174e4772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:07.521890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a253b-755d-4261-a5e7-542f174e4772 None admin] RBAC: Adding query filter params (role.id=3320a9786ec948aab8c8fcbb156eb9fa, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c, user.id=142d81749cb3486b87b11184e2cd8b6e) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:07.522194 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a253b-755d-4261-a5e7-542f174e4772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:07.523680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a253b-755d-4261-a5e7-542f174e4772 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:07.530951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 109/208] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:07 2022] GET /identity/v3/role_assignments?user.id=142d81749cb3486b87b11184e2cd8b6e&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=3320a9786ec948aab8c8fcbb156eb9fa => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:08.615065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edf376-51f7-4f70-8215-45a1586ea994 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:08.615577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edf376-51f7-4f70-8215-45a1586ea994 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:08.615821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edf376-51f7-4f70-8215-45a1586ea994 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:08.617276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 100/209] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:08.623382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685388e-94c9-4893-a23e-9405120e85ea None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:08.623824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685388e-94c9-4893-a23e-9405120e85ea None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:08.624113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685388e-94c9-4893-a23e-9405120e85ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:08.643592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5685388e-94c9-4893-a23e-9405120e85ea None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:08.644836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5685388e-94c9-4893-a23e-9405120e85ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:08.706414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867245-3819-4fc7-9a0d-db1a8e609ebc None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:08.706729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867245-3819-4fc7-9a0d-db1a8e609ebc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:08.707054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867245-3819-4fc7-9a0d-db1a8e609ebc None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:08.708641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 101/210] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:08.716378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca39acd-6a6e-4076-ac2f-33bce86c95df None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:08.716699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca39acd-6a6e-4076-ac2f-33bce86c95df None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:08.716927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca39acd-6a6e-4076-ac2f-33bce86c95df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:08.729334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 110/211] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:08.737754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1ca39acd-6a6e-4076-ac2f-33bce86c95df None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:08.739244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1ca39acd-6a6e-4076-ac2f-33bce86c95df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:08.768259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:08.819130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 102/212] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:09.012889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685388e-94c9-4893-a23e-9405120e85ea None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:09.099377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca39acd-6a6e-4076-ac2f-33bce86c95df None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:09.128036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685388e-94c9-4893-a23e-9405120e85ea None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:09.151026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:09.169736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a88b4b-a502-4701-af6f-db68334aea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:09.170810 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a88b4b-a502-4701-af6f-db68334aea8c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.171333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a88b4b-a502-4701-af6f-db68334aea8c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.171812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a88b4b-a502-4701-af6f-db68334aea8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.183502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a88b4b-a502-4701-af6f-db68334aea8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e6a88b4b-a502-4701-af6f-db68334aea8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:09.189025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:09.189889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:09.190665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:09.198802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 111/213] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:09.200027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a88b4b-a502-4701-af6f-db68334aea8c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:09.219138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca39acd-6a6e-4076-ac2f-33bce86c95df None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:09.262050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de7d8f-3d83-42af-92f2-979adcd49f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:09.263102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7d8f-3d83-42af-92f2-979adcd49f89 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.263475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7d8f-3d83-42af-92f2-979adcd49f89 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.263729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7d8f-3d83-42af-92f2-979adcd49f89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.264839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de7d8f-3d83-42af-92f2-979adcd49f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:09.266368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de7d8f-3d83-42af-92f2-979adcd49f89 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:09.270970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 103/214] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:09.270970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae75d5-2454-4243-9c9f-e5ccd771a39f None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.270970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae75d5-2454-4243-9c9f-e5ccd771a39f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.270970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae75d5-2454-4243-9c9f-e5ccd771a39f None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.271321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 104/215] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:09.275856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae75d5-2454-4243-9c9f-e5ccd771a39f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:09.313891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:09.325269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a88b4b-a502-4701-af6f-db68334aea8c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:09.372413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350dfbe-027d-486b-b3c4-8ad1e41d52b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:09.373307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350dfbe-027d-486b-b3c4-8ad1e41d52b0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.373594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350dfbe-027d-486b-b3c4-8ad1e41d52b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.373869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350dfbe-027d-486b-b3c4-8ad1e41d52b0 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.374539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350dfbe-027d-486b-b3c4-8ad1e41d52b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:09.374838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350dfbe-027d-486b-b3c4-8ad1e41d52b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:09.376279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350dfbe-027d-486b-b3c4-8ad1e41d52b0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:09.386906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 112/216] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:09.388359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95c1db-b91e-455e-978b-f3c22d3ccc1a None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.388647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95c1db-b91e-455e-978b-f3c22d3ccc1a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.388921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95c1db-b91e-455e-978b-f3c22d3ccc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.415543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6d95c1db-b91e-455e-978b-f3c22d3ccc1a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:09.417057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6d95c1db-b91e-455e-978b-f3c22d3ccc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:09.420126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae75d5-2454-4243-9c9f-e5ccd771a39f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:09.467110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d55ab-5117-46a4-bac8-7297343f185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:09.468053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d55ab-5117-46a4-bac8-7297343f185e None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.468053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d55ab-5117-46a4-bac8-7297343f185e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.468250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d55ab-5117-46a4-bac8-7297343f185e None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.469214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d55ab-5117-46a4-bac8-7297343f185e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:09.470369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d55ab-5117-46a4-bac8-7297343f185e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:09.474793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:09.510809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 113/217] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:09.513284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95c1db-b91e-455e-978b-f3c22d3ccc1a None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:09.594962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 105/218] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:09 2022] POST /identity/v3/users => generated 297 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:09.594962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a9d0-f76e-4d32-b35d-d4d498af8718 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.594962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a9d0-f76e-4d32-b35d-d4d498af8718 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.594962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a9d0-f76e-4d32-b35d-d4d498af8718 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.594962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 106/219] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:09.602951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741f32-9b1f-40bd-a5f6-c1191be9d6dc None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.602951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741f32-9b1f-40bd-a5f6-c1191be9d6dc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.602951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741f32-9b1f-40bd-a5f6-c1191be9d6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.644574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-82741f32-9b1f-40bd-a5f6-c1191be9d6dc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:09.646078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-82741f32-9b1f-40bd-a5f6-c1191be9d6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:09.710828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95c1db-b91e-455e-978b-f3c22d3ccc1a None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:09.771533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdbd16-1f22-40d8-808e-070c89473365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:09.772502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 107/220] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:09.775024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdbd16-1f22-40d8-808e-070c89473365 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.775499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdbd16-1f22-40d8-808e-070c89473365 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.775933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdbd16-1f22-40d8-808e-070c89473365 None admin] PATH_INFO: `/v3/users/2d43bf0e9493437e85fd8f8e636ed940` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.797532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdbd16-1f22-40d8-808e-070c89473365 None admin] RBAC: Authorizing `identity:get_user(user_id=2d43bf0e9493437e85fd8f8e636ed940)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:09.799392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdbd16-1f22-40d8-808e-070c89473365 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:09.801914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 114/221] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:09 2022] GET /identity/v3/users/2d43bf0e9493437e85fd8f8e636ed940 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:09.808394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82741f32-9b1f-40bd-a5f6-c1191be9d6dc None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:09.811230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdbd16-1f22-40d8-808e-070c89473365 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:09.833334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:09.856394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:09.949214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdbd16-1f22-40d8-808e-070c89473365 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:09.953967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82741f32-9b1f-40bd-a5f6-c1191be9d6dc None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:09.992321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2083b2-8372-4038-89a2-f2b06016e13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:09.993219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2083b2-8372-4038-89a2-f2b06016e13f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.993586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2083b2-8372-4038-89a2-f2b06016e13f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.993867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2083b2-8372-4038-89a2-f2b06016e13f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:09.998361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8f0ca-b685-4687-b14a-7cc40778f100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:09.999168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8f0ca-b685-4687-b14a-7cc40778f100 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:09.999547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8f0ca-b685-4687-b14a-7cc40778f100 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:09.999917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8f0ca-b685-4687-b14a-7cc40778f100 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.007466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2083b2-8372-4038-89a2-f2b06016e13f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-7f2083b2-8372-4038-89a2-f2b06016e13f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:10.012857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:10.013740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:10.014546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:10.017244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 115/222] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:10.023549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2083b2-8372-4038-89a2-f2b06016e13f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.089288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8f0ca-b685-4687-b14a-7cc40778f100 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.091898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8f0ca-b685-4687-b14a-7cc40778f100 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.116465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.135392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.150028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2083b2-8372-4038-89a2-f2b06016e13f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.186735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.187431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.187712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.187984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.188662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:10.188961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.190348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.200201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 116/223] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:10.201132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.219211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 108/224] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:35:10.220348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8f0ca-b685-4687-b14a-7cc40778f100 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.332420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8f0ca-b685-4687-b14a-7cc40778f100 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.335809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f43950-25c3-4c9b-937f-3c79ea107c2c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.351261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.376866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6cd55-4ee4-4f17-82ca-b7ec1e74c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.377607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6cd55-4ee4-4f17-82ca-b7ec1e74c077 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.378081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6cd55-4ee4-4f17-82ca-b7ec1e74c077 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.378416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815e21e-4290-4404-a23f-c9b794eb3a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.378532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6cd55-4ee4-4f17-82ca-b7ec1e74c077 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.379255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815e21e-4290-4404-a23f-c9b794eb3a91 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.379639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815e21e-4290-4404-a23f-c9b794eb3a91 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.379986 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815e21e-4290-4404-a23f-c9b794eb3a91 None admin] PATH_INFO: `/v3/users/59b49efadab246e1aec3f04209f16263` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.382528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815e21e-4290-4404-a23f-c9b794eb3a91 None admin] RBAC: Authorizing `identity:get_user(user_id=59b49efadab246e1aec3f04209f16263)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.384117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815e21e-4290-4404-a23f-c9b794eb3a91 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.386558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 109/225] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/users/59b49efadab246e1aec3f04209f16263 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:10.387717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815e21e-4290-4404-a23f-c9b794eb3a91 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.391480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b6cd55-4ee4-4f17-82ca-b7ec1e74c077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-23b6cd55-4ee4-4f17-82ca-b7ec1e74c077 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:10.398508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:10.404080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 117/226] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:10.404879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6cd55-4ee4-4f17-82ca-b7ec1e74c077 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.423871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.514279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815e21e-4290-4404-a23f-c9b794eb3a91 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.530833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6cd55-4ee4-4f17-82ca-b7ec1e74c077 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.558893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e86c6-2619-4b7d-8fdd-5296535c92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.563296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e86c6-2619-4b7d-8fdd-5296535c92f2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.563586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e86c6-2619-4b7d-8fdd-5296535c92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.563864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e86c6-2619-4b7d-8fdd-5296535c92f2 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.565800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e86c6-2619-4b7d-8fdd-5296535c92f2 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.567327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e86c6-2619-4b7d-8fdd-5296535c92f2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.569166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 110/227] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:10.569756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e86c6-2619-4b7d-8fdd-5296535c92f2 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.582134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec93b2-77d3-4650-a608-2ab7a6161770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.583138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec93b2-77d3-4650-a608-2ab7a6161770 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.583640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec93b2-77d3-4650-a608-2ab7a6161770 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.584113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec93b2-77d3-4650-a608-2ab7a6161770 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.585513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec93b2-77d3-4650-a608-2ab7a6161770 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.587214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec93b2-77d3-4650-a608-2ab7a6161770 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.589061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 118/228] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:10.591092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec93b2-77d3-4650-a608-2ab7a6161770 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.641585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.678593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.694938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e86c6-2619-4b7d-8fdd-5296535c92f2 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.700128 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec93b2-77d3-4650-a608-2ab7a6161770 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.738305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.739792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.740226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.740778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.741770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] RBAC: Adding query filter params (role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=f618dc420f704f2c8a85cfe182a07300, user.id=2d43bf0e9493437e85fd8f8e636ed940) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:10.742030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.742390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.743162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.743539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.743911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.744676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:10.744845 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.745157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.746661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:10.751574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 119/229] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/role_assignments?user.id=2d43bf0e9493437e85fd8f8e636ed940&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:10.752584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.758242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 111/230] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:10.764249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:10.942671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87e929-7051-40e8-9df9-094a7b35ae43 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.961899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473be12-fe8d-440e-b14f-75e501229343 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:10.982111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:10.992290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbbc72-98ef-43c8-8da5-9d9c97623cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:10.993226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbbc72-98ef-43c8-8da5-9d9c97623cba None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:10.993504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbbc72-98ef-43c8-8da5-9d9c97623cba None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:10.993786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbbc72-98ef-43c8-8da5-9d9c97623cba None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300/users/59b49efadab246e1aec3f04209f16263/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:10.996867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbbc72-98ef-43c8-8da5-9d9c97623cba None admin] RBAC: Authorizing `identity:create_grant(project_id=f618dc420f704f2c8a85cfe182a07300, user_id=59b49efadab246e1aec3f04209f16263, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:10.998328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbbc72-98ef-43c8-8da5-9d9c97623cba None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.010416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385b3f5-4070-4551-8695-6a7d39573f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:11.012350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385b3f5-4070-4551-8695-6a7d39573f16 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:11.013055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385b3f5-4070-4551-8695-6a7d39573f16 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:11.013525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385b3f5-4070-4551-8695-6a7d39573f16 None admin] PATH_INFO: `/v3/users/59b49efadab246e1aec3f04209f16263` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:11.015116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385b3f5-4070-4551-8695-6a7d39573f16 None admin] RBAC: Authorizing `identity:get_user(user_id=59b49efadab246e1aec3f04209f16263)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:11.017089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385b3f5-4070-4551-8695-6a7d39573f16 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.019216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 112/231] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:10 2022] GET /identity/v3/users/59b49efadab246e1aec3f04209f16263 => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:11.025318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385b3f5-4070-4551-8695-6a7d39573f16 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:11.079898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 120/232] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:35:10 2022] PUT /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300/users/59b49efadab246e1aec3f04209f16263/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:11.151172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385b3f5-4070-4551-8695-6a7d39573f16 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:11.191099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1576-e4b1-4a09-8a84-44acd9cec445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:11.191775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1576-e4b1-4a09-8a84-44acd9cec445 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:11.192050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1576-e4b1-4a09-8a84-44acd9cec445 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:11.192321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1576-e4b1-4a09-8a84-44acd9cec445 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:11.193649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b1576-e4b1-4a09-8a84-44acd9cec445 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:11.195122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b1576-e4b1-4a09-8a84-44acd9cec445 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.196852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 113/233] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:11 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:11.203424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbbc72-98ef-43c8-8da5-9d9c97623cba None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:11.207768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:11.338398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbbc72-98ef-43c8-8da5-9d9c97623cba None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:11.376197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900c7ee-09b5-4ef5-b3cb-7a615daf5f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:11.376925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c7ee-09b5-4ef5-b3cb-7a615daf5f63 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:11.377209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c7ee-09b5-4ef5-b3cb-7a615daf5f63 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:11.377484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c7ee-09b5-4ef5-b3cb-7a615daf5f63 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300/users/59b49efadab246e1aec3f04209f16263/roles/98aaf5af7bb24b1596a9b01fd76eaf2c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:11.379754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900c7ee-09b5-4ef5-b3cb-7a615daf5f63 None admin] RBAC: Authorizing `identity:create_grant(project_id=f618dc420f704f2c8a85cfe182a07300, user_id=59b49efadab246e1aec3f04209f16263, role_id=98aaf5af7bb24b1596a9b01fd76eaf2c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:11.381296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900c7ee-09b5-4ef5-b3cb-7a615daf5f63 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:11.489074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 121/234] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:35:11 2022] PUT /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300/users/59b49efadab246e1aec3f04209f16263/roles/98aaf5af7bb24b1596a9b01fd76eaf2c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:12.004950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c11f6-d5ff-44f7-a380-7808f709fbc7 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.005459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c11f6-d5ff-44f7-a380-7808f709fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.005459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c11f6-d5ff-44f7-a380-7808f709fbc7 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.007056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 114/235] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:12.012461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8e66-d574-4ea8-9327-a1d264b5d578 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.012902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8e66-d574-4ea8-9327-a1d264b5d578 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.012978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8e66-d574-4ea8-9327-a1d264b5d578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.038979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-86fa8e66-d574-4ea8-9327-a1d264b5d578 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:12.040170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-86fa8e66-d574-4ea8-9327-a1d264b5d578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:12.121655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc5890-4ed6-44ba-8767-1162c9203595 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.121975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc5890-4ed6-44ba-8767-1162c9203595 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.122191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc5890-4ed6-44ba-8767-1162c9203595 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.123701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 115/236] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:12.129189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83b167-5430-4485-b75b-b318a9107033 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.129735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83b167-5430-4485-b75b-b318a9107033 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.130200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83b167-5430-4485-b75b-b318a9107033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.132926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:12.153778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 122/237] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:12.154705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1c83b167-5430-4485-b75b-b318a9107033 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:12.160006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1c83b167-5430-4485-b75b-b318a9107033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:12.234619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:12.257003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 116/238] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:12.491622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa8e66-d574-4ea8-9327-a1d264b5d578 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:12.610472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa8e66-d574-4ea8-9327-a1d264b5d578 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:12.646499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954072bd-be48-4218-9723-3c5126ea43d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:12.647624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954072bd-be48-4218-9723-3c5126ea43d4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.648100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954072bd-be48-4218-9723-3c5126ea43d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.648564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954072bd-be48-4218-9723-3c5126ea43d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.659950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954072bd-be48-4218-9723-3c5126ea43d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-954072bd-be48-4218-9723-3c5126ea43d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:12.665133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:12.665975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:12.666744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:12.669531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 123/239] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:12.680031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83b167-5430-4485-b75b-b318a9107033 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:12.738967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954072bd-be48-4218-9723-3c5126ea43d4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:12.819280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:12.863372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83b167-5430-4485-b75b-b318a9107033 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:12.901449 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954072bd-be48-4218-9723-3c5126ea43d4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:12.926429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:12.927079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.927475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.927741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.928419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:12.928715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:12.930096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:12.943717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab561f95-e6e2-40ec-80e4-4a7e932eaf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:12.944635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561f95-e6e2-40ec-80e4-4a7e932eaf3c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:12.945079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561f95-e6e2-40ec-80e4-4a7e932eaf3c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:12.945535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561f95-e6e2-40ec-80e4-4a7e932eaf3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:12.946822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab561f95-e6e2-40ec-80e4-4a7e932eaf3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:12.949141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab561f95-e6e2-40ec-80e4-4a7e932eaf3c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:12.950383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 117/240] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:12.951119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 124/241] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:12.955907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:12.959539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab561f95-e6e2-40ec-80e4-4a7e932eaf3c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:13.038638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:13.112386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4af96-08e9-4df0-ad45-b092af7a7e14 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:13.158690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843d919-80c2-4a2b-a2e0-aa713b41cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:13.159287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843d919-80c2-4a2b-a2e0-aa713b41cf28 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:13.159564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843d919-80c2-4a2b-a2e0-aa713b41cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:13.159833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843d919-80c2-4a2b-a2e0-aa713b41cf28 None admin] PATH_INFO: `/v3/users/3aadc0e890a24843a87870a13a9ca70e` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:13.178041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843d919-80c2-4a2b-a2e0-aa713b41cf28 None admin] RBAC: Authorizing `identity:get_user(user_id=3aadc0e890a24843a87870a13a9ca70e)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:13.179728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843d919-80c2-4a2b-a2e0-aa713b41cf28 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:13.185146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 118/242] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:12 2022] GET /identity/v3/users/3aadc0e890a24843a87870a13a9ca70e => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:13.190560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843d919-80c2-4a2b-a2e0-aa713b41cf28 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:13.195327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab561f95-e6e2-40ec-80e4-4a7e932eaf3c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:13.248415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:13.279655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ff92c-cc03-41a8-83d7-ddeebaa7a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:13.280389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff92c-cc03-41a8-83d7-ddeebaa7a0ee None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:13.280673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff92c-cc03-41a8-83d7-ddeebaa7a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:13.280948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff92c-cc03-41a8-83d7-ddeebaa7a0ee None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:13.281656 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff92c-cc03-41a8-83d7-ddeebaa7a0ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:13.283361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff92c-cc03-41a8-83d7-ddeebaa7a0ee None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:13.312594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843d919-80c2-4a2b-a2e0-aa713b41cf28 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:13.314651 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:13.347820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760c992-2a36-4431-bb6c-1ec923a345c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:13.348642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c992-2a36-4431-bb6c-1ec923a345c0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:13.348922 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c992-2a36-4431-bb6c-1ec923a345c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:13.349191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c992-2a36-4431-bb6c-1ec923a345c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:13.383854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 125/243] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:12 2022] POST /identity/v3/users => generated 293 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:13.390947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b8242-3b1b-49fe-9bf8-e7faf7536dcb None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:13.390947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b8242-3b1b-49fe-9bf8-e7faf7536dcb None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:13.390947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b8242-3b1b-49fe-9bf8-e7faf7536dcb None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:13.390947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 126/244] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:13.400703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348114c-34c6-45f4-878b-afffd03dfbfc None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:13.401056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348114c-34c6-45f4-878b-afffd03dfbfc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:13.401245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348114c-34c6-45f4-878b-afffd03dfbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:13.443628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-a348114c-34c6-45f4-878b-afffd03dfbfc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:13.446168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a348114c-34c6-45f4-878b-afffd03dfbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:13.483175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760c992-2a36-4431-bb6c-1ec923a345c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:13.484928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760c992-2a36-4431-bb6c-1ec923a345c0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:13.587507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 127/245] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:13.589918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef2f3a-14ff-48dc-93b3-63615810e10e None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:13.590208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef2f3a-14ff-48dc-93b3-63615810e10e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:13.590745 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef2f3a-14ff-48dc-93b3-63615810e10e None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:13.592499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 128/246] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:13.597972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7300-7366-4d45-a9ab-55037a1bf7c8 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:13.598251 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7300-7366-4d45-a9ab-55037a1bf7c8 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:13.598521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7300-7366-4d45-a9ab-55037a1bf7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:13.629430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-13dd7300-7366-4d45-a9ab-55037a1bf7c8 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:13.631040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-13dd7300-7366-4d45-a9ab-55037a1bf7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:13.674845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 119/247] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:13.682307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760c992-2a36-4431-bb6c-1ec923a345c0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:13.685161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:13.759030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 129/248] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:13.841983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760c992-2a36-4431-bb6c-1ec923a345c0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:13.894063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd7300-7366-4d45-a9ab-55037a1bf7c8 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:13.902832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dc37a-5e86-40e5-b034-f7aceba5e4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:13.903589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc37a-5e86-40e5-b034-f7aceba5e4a3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:13.904018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc37a-5e86-40e5-b034-f7aceba5e4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:13.904449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dc37a-5e86-40e5-b034-f7aceba5e4a3 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:13.906430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dc37a-5e86-40e5-b034-f7aceba5e4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:13.908035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dc37a-5e86-40e5-b034-f7aceba5e4a3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:13.910019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 120/249] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:13 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:13.920221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dc37a-5e86-40e5-b034-f7aceba5e4a3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:13.955975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:14.035828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:14.041807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd7300-7366-4d45-a9ab-55037a1bf7c8 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.081003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dc37a-5e86-40e5-b034-f7aceba5e4a3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.090562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482c779-dfe4-4174-b777-cb6a54684060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.091249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482c779-dfe4-4174-b777-cb6a54684060 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.091528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482c779-dfe4-4174-b777-cb6a54684060 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.091798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482c779-dfe4-4174-b777-cb6a54684060 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.112606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482c779-dfe4-4174-b777-cb6a54684060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-a482c779-dfe4-4174-b777-cb6a54684060 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:14.117824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:14.118859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:14.119682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 130/250] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:14.126238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482c779-dfe4-4174-b777-cb6a54684060 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.150886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.152546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.153198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.153198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.153839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] RBAC: Adding query filter params (role.id=3320a9786ec948aab8c8fcbb156eb9fa, scope.project.id=f618dc420f704f2c8a85cfe182a07300, user.id=3aadc0e890a24843a87870a13a9ca70e) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:14.154144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:14.155659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:14.162392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 121/251] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:13 2022] GET /identity/v3/role_assignments?user.id=3aadc0e890a24843a87870a13a9ca70e&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=3320a9786ec948aab8c8fcbb156eb9fa => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:14.163543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.294970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97f426-dfa9-4f98-99b2-c46c86a38923 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.315252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:14.326854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482c779-dfe4-4174-b777-cb6a54684060 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.336673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414aabe8-d013-4ce5-b141-b8edb21a1678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.337387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aabe8-d013-4ce5-b141-b8edb21a1678 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.337666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aabe8-d013-4ce5-b141-b8edb21a1678 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.338003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aabe8-d013-4ce5-b141-b8edb21a1678 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.340490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:14.351826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414aabe8-d013-4ce5-b141-b8edb21a1678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-414aabe8-d013-4ce5-b141-b8edb21a1678 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:14.357413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:14.359176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:14.359946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 122/252] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:14.364330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b453da-590e-4efc-b987-b00245d07645 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.364752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b453da-590e-4efc-b987-b00245d07645 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.365164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b453da-590e-4efc-b987-b00245d07645 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.366483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 123/253] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:14.368615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b453da-590e-4efc-b987-b00245d07645 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.377783 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecd02e-4d68-4e9e-a4d3-fa84f100edf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.378675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd02e-4d68-4e9e-a4d3-fa84f100edf6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.379167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd02e-4d68-4e9e-a4d3-fa84f100edf6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.379599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd02e-4d68-4e9e-a4d3-fa84f100edf6 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.380461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecd02e-4d68-4e9e-a4d3-fa84f100edf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:14.380939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecd02e-4d68-4e9e-a4d3-fa84f100edf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:14.382498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecd02e-4d68-4e9e-a4d3-fa84f100edf6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:14.393018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 131/254] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:14.398981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeb9cb-074a-4efc-9804-a6a730d31569 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.399836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeb9cb-074a-4efc-9804-a6a730d31569 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.400306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeb9cb-074a-4efc-9804-a6a730d31569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.421528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8eeeb9cb-074a-4efc-9804-a6a730d31569 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:14.423032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8eeeb9cb-074a-4efc-9804-a6a730d31569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:14.497850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 132/255] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:14.499048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeb9cb-074a-4efc-9804-a6a730d31569 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.600704 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:14.607016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeb9cb-074a-4efc-9804-a6a730d31569 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.628223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b453da-590e-4efc-b987-b00245d07645 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.644235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5847e0-9628-4ce8-bed2-de58604ed793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.645027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5847e0-9628-4ce8-bed2-de58604ed793 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.645399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5847e0-9628-4ce8-bed2-de58604ed793 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.645850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5847e0-9628-4ce8-bed2-de58604ed793 None admin] PATH_INFO: `/v3/users/2d43bf0e9493437e85fd8f8e636ed940` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.647035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5847e0-9628-4ce8-bed2-de58604ed793 None admin] RBAC: Authorizing `identity:get_user(user_id=2d43bf0e9493437e85fd8f8e636ed940)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:14.648595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5847e0-9628-4ce8-bed2-de58604ed793 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:14.650299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 133/256] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:14 2022] GET /identity/v3/users/2d43bf0e9493437e85fd8f8e636ed940 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:14.656861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5847e0-9628-4ce8-bed2-de58604ed793 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.779690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.780327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.780605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.780872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.781549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:14.781847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:14.783932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5847e0-9628-4ce8-bed2-de58604ed793 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.787315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:14.812870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 124/257] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:14.821864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ee9a3-00de-4003-a7d0-9c13b5eb0226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.822506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ee9a3-00de-4003-a7d0-9c13b5eb0226 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.822784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ee9a3-00de-4003-a7d0-9c13b5eb0226 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.823078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ee9a3-00de-4003-a7d0-9c13b5eb0226 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.824193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.824305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ee9a3-00de-4003-a7d0-9c13b5eb0226 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:14.825850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ee9a3-00de-4003-a7d0-9c13b5eb0226 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:14.827475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 134/258] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:14 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:14.833066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ee9a3-00de-4003-a7d0-9c13b5eb0226 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:14.863283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:14.943912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ee9a3-00de-4003-a7d0-9c13b5eb0226 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:14.974062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:14.980217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033145ca-b120-4e3f-8315-eeeed474ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:14.980942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033145ca-b120-4e3f-8315-eeeed474ee7f None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:14.981216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033145ca-b120-4e3f-8315-eeeed474ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:14.981485 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033145ca-b120-4e3f-8315-eeeed474ee7f None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300/users/2d43bf0e9493437e85fd8f8e636ed940/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:14.983522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033145ca-b120-4e3f-8315-eeeed474ee7f None admin] RBAC: Authorizing `identity:create_grant(project_id=f618dc420f704f2c8a85cfe182a07300, user_id=2d43bf0e9493437e85fd8f8e636ed940, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:14.985033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033145ca-b120-4e3f-8315-eeeed474ee7f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.031069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3198a-4cd1-4673-bbd3-a512cfe9cc77 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.061487 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 135/259] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:35:14 2022] PUT /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300/users/2d43bf0e9493437e85fd8f8e636ed940/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:35:15.063220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033145ca-b120-4e3f-8315-eeeed474ee7f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.080163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6d8e-04b0-47d4-a632-840c61d67e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.080807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6d8e-04b0-47d4-a632-840c61d67e88 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.081083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6d8e-04b0-47d4-a632-840c61d67e88 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.081638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6d8e-04b0-47d4-a632-840c61d67e88 None admin] PATH_INFO: `/v3/users/59b49efadab246e1aec3f04209f16263` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.084864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab6d8e-04b0-47d4-a632-840c61d67e88 None admin] RBAC: Authorizing `identity:get_user(user_id=59b49efadab246e1aec3f04209f16263)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.086687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab6d8e-04b0-47d4-a632-840c61d67e88 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.088760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 125/260] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:14 2022] GET /identity/v3/users/59b49efadab246e1aec3f04209f16263 => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:15.094705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6d8e-04b0-47d4-a632-840c61d67e88 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.272475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6d8e-04b0-47d4-a632-840c61d67e88 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.300221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.313878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc7e23-290e-4e45-8884-a3b25e5aed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.314563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc7e23-290e-4e45-8884-a3b25e5aed4e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.314835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc7e23-290e-4e45-8884-a3b25e5aed4e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.315845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc7e23-290e-4e45-8884-a3b25e5aed4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.345667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.383751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033145ca-b120-4e3f-8315-eeeed474ee7f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.392790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc7e23-290e-4e45-8884-a3b25e5aed4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.394253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc7e23-290e-4e45-8884-a3b25e5aed4e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.502112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 126/261] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:35:15.507899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc7e23-290e-4e45-8884-a3b25e5aed4e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.512296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d69f12-d024-4129-8ec7-3f1a10b3751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.513056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69f12-d024-4129-8ec7-3f1a10b3751f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.513335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69f12-d024-4129-8ec7-3f1a10b3751f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.513606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69f12-d024-4129-8ec7-3f1a10b3751f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.535304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.559615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d69f12-d024-4129-8ec7-3f1a10b3751f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-74d69f12-d024-4129-8ec7-3f1a10b3751f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:15.564523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:15.565617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:15.566366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 136/262] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:15.583786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d69f12-d024-4129-8ec7-3f1a10b3751f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.629070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc7e23-290e-4e45-8884-a3b25e5aed4e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.668228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e5aeb-c6e4-4731-8297-ba653f7e8aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.668934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e5aeb-c6e4-4731-8297-ba653f7e8aee None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.669212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e5aeb-c6e4-4731-8297-ba653f7e8aee None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.669481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e5aeb-c6e4-4731-8297-ba653f7e8aee None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.670675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e5aeb-c6e4-4731-8297-ba653f7e8aee None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.672230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e5aeb-c6e4-4731-8297-ba653f7e8aee None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.673864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 127/263] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:15.681861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e5aeb-c6e4-4731-8297-ba653f7e8aee None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:15.790576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:15.794203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e5aeb-c6e4-4731-8297-ba653f7e8aee None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:15.837004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:15.837625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:15.837898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:15.838158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:15.839505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=f618dc420f704f2c8a85cfe182a07300, user.id=59b49efadab246e1aec3f04209f16263) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:15.839781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:15.841489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:15.848593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 128/264] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/role_assignments?user.id=59b49efadab246e1aec3f04209f16263&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:15.950248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d69f12-d024-4129-8ec7-3f1a10b3751f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.002824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.003467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.003746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.004014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.004683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:16.004978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.006342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.020141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 137/265] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:16.025694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.147558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b386d8-a20c-4c6d-a0ff-53b786897208 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.190200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16b38a-bd8e-417d-a5a7-744ab9b2db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.191257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16b38a-bd8e-417d-a5a7-744ab9b2db47 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.191697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16b38a-bd8e-417d-a5a7-744ab9b2db47 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.192139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16b38a-bd8e-417d-a5a7-744ab9b2db47 None admin] PATH_INFO: `/v3/users/59b49efadab246e1aec3f04209f16263` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.193419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16b38a-bd8e-417d-a5a7-744ab9b2db47 None admin] RBAC: Authorizing `identity:get_user(user_id=59b49efadab246e1aec3f04209f16263)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.194871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16b38a-bd8e-417d-a5a7-744ab9b2db47 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.196769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 129/266] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:16 2022] GET /identity/v3/users/59b49efadab246e1aec3f04209f16263 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:16.202980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.220902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:16.228571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a543a-550c-4385-9f6a-2790624da6ef None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.228849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a543a-550c-4385-9f6a-2790624da6ef None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.229116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a543a-550c-4385-9f6a-2790624da6ef None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.240580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 130/267] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:16.240580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7383-b16b-4379-845a-677a9d8167bb None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.240580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7383-b16b-4379-845a-677a9d8167bb None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.240580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7383-b16b-4379-845a-677a9d8167bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.259278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7acb7383-b16b-4379-845a-677a9d8167bb None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:16.260591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7acb7383-b16b-4379-845a-677a9d8167bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:16.282517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:16.344816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 131/268] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:16.385016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d46ac2-0ac8-467b-a519-b49325e9b3a6 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.428739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda497e9-560a-49d1-b3cd-60464120643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.429828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda497e9-560a-49d1-b3cd-60464120643b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.430342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda497e9-560a-49d1-b3cd-60464120643b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.430863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda497e9-560a-49d1-b3cd-60464120643b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.512636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda497e9-560a-49d1-b3cd-60464120643b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.514432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda497e9-560a-49d1-b3cd-60464120643b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.525461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:16.623816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 138/269] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:16.632146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb7383-b16b-4379-845a-677a9d8167bb None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.765821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda497e9-560a-49d1-b3cd-60464120643b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.788735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb7383-b16b-4379-845a-677a9d8167bb None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.853301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061019da-de72-43e0-98b2-08916ea1167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.853960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061019da-de72-43e0-98b2-08916ea1167c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.854235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061019da-de72-43e0-98b2-08916ea1167c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.855163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061019da-de72-43e0-98b2-08916ea1167c None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.856426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061019da-de72-43e0-98b2-08916ea1167c None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.857940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061019da-de72-43e0-98b2-08916ea1167c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.859952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 132/270] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:16 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:16.861159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145ab2b-9583-4169-89ae-facefbb428ba None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.861441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145ab2b-9583-4169-89ae-facefbb428ba None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.861710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145ab2b-9583-4169-89ae-facefbb428ba None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.867041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 133/271] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:16.868187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145ab2b-9583-4169-89ae-facefbb428ba None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:16.893361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:16.897812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda497e9-560a-49d1-b3cd-60464120643b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:16.908806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:16.936846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d3af1-8b9e-416b-8cc8-dced137a21b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:16.937575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d3af1-8b9e-416b-8cc8-dced137a21b6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.937853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d3af1-8b9e-416b-8cc8-dced137a21b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.938121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d3af1-8b9e-416b-8cc8-dced137a21b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.939450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d3af1-8b9e-416b-8cc8-dced137a21b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:16.941094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d3af1-8b9e-416b-8cc8-dced137a21b6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:16.942881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 139/272] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:16.944003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa4738-e698-4493-ab8f-71fa79d7a2d6 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:16.944321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa4738-e698-4493-ab8f-71fa79d7a2d6 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:16.944623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa4738-e698-4493-ab8f-71fa79d7a2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:16.965897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7cfa4738-e698-4493-ab8f-71fa79d7a2d6 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:16.967268 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7cfa4738-e698-4493-ab8f-71fa79d7a2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:16.999526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145ab2b-9583-4169-89ae-facefbb428ba None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:17.042861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 140/273] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:17.044067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa4738-e698-4493-ab8f-71fa79d7a2d6 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:17.050044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cba5ad-a3cf-4cc1-9088-fa39b2f00378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:17.050714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cba5ad-a3cf-4cc1-9088-fa39b2f00378 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.051033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cba5ad-a3cf-4cc1-9088-fa39b2f00378 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.052217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cba5ad-a3cf-4cc1-9088-fa39b2f00378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.057234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cba5ad-a3cf-4cc1-9088-fa39b2f00378 None admin] RBAC: Adding query filter params (role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=f618dc420f704f2c8a85cfe182a07300, user.id=59b49efadab246e1aec3f04209f16263) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:17.057559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cba5ad-a3cf-4cc1-9088-fa39b2f00378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:17.061316 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cba5ad-a3cf-4cc1-9088-fa39b2f00378 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:17.072471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 134/274] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:16 2022] GET /identity/v3/role_assignments?user.id=59b49efadab246e1aec3f04209f16263&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:17.152091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7f098-f6ae-4f27-85db-e799d968057e None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.153035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7f098-f6ae-4f27-85db-e799d968057e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.153314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7f098-f6ae-4f27-85db-e799d968057e None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.155129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 135/275] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:17.164540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbec155-eafc-4d02-a998-070e13f2e5ee None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.164875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbec155-eafc-4d02-a998-070e13f2e5ee None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.165073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbec155-eafc-4d02-a998-070e13f2e5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.186454 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1cbec155-eafc-4d02-a998-070e13f2e5ee None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:17.187895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1cbec155-eafc-4d02-a998-070e13f2e5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:17.209237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa4738-e698-4493-ab8f-71fa79d7a2d6 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:17.223541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:17.271659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e9799-0d94-4260-a9e1-624837004252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:17.271659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e9799-0d94-4260-a9e1-624837004252 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.271935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e9799-0d94-4260-a9e1-624837004252 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.272203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e9799-0d94-4260-a9e1-624837004252 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.272852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e9799-0d94-4260-a9e1-624837004252 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:17.274459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e9799-0d94-4260-a9e1-624837004252 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:17.287323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 136/276] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:17.379680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbec155-eafc-4d02-a998-070e13f2e5ee None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:17.388149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 141/277] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:17 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:17.400050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:17.503018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbec155-eafc-4d02-a998-070e13f2e5ee None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:17.542691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3388aa4-b864-4c9f-ae94-22c385580b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:17.543485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3388aa4-b864-4c9f-ae94-22c385580b0e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.543762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3388aa4-b864-4c9f-ae94-22c385580b0e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.544032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3388aa4-b864-4c9f-ae94-22c385580b0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.555393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3388aa4-b864-4c9f-ae94-22c385580b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-e3388aa4-b864-4c9f-ae94-22c385580b0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:17.563388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:17.564300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:17.565133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 137/278] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:17.568238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e9799-0d94-4260-a9e1-624837004252 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:17.568376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3388aa4-b864-4c9f-ae94-22c385580b0e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:17.673155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:17.701205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3388aa4-b864-4c9f-ae94-22c385580b0e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:17.719800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e9799-0d94-4260-a9e1-624837004252 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:17.749493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:17.750471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.750993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.751441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.752213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] RBAC: Adding query filter params (name=member) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:17.752612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:17.754090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:17.766398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd25a4-1bd8-49af-9d00-2c139b86339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:17.767504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd25a4-1bd8-49af-9d00-2c139b86339c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.767612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 138/279] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:17.768072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd25a4-1bd8-49af-9d00-2c139b86339c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.768519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd25a4-1bd8-49af-9d00-2c139b86339c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.773921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:17.785285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd25a4-1bd8-49af-9d00-2c139b86339c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-83fd25a4-1bd8-49af-9d00-2c139b86339c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:17.791898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:17.792984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:17.793984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:17.795987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 142/280] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:17.803478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd25a4-1bd8-49af-9d00-2c139b86339c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:17.837993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:17.913849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c85c12-43d5-458f-9428-25a1b7872d63 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:17.949956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd25a4-1bd8-49af-9d00-2c139b86339c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:17.956705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a7160-9968-4f90-bbc4-92eecae0b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:17.957354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7160-9968-4f90-bbc4-92eecae0b403 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.957635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7160-9968-4f90-bbc4-92eecae0b403 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.957904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7160-9968-4f90-bbc4-92eecae0b403 None admin] PATH_INFO: `/v3/users/0bd1f25b3d704ffd8f1d4f34e0ff8bd9` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.967669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:17.974030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a7160-9968-4f90-bbc4-92eecae0b403 None admin] RBAC: Authorizing `identity:get_user(user_id=0bd1f25b3d704ffd8f1d4f34e0ff8bd9)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:17.975587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a7160-9968-4f90-bbc4-92eecae0b403 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:17.977326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 139/281] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:17 2022] GET /identity/v3/users/0bd1f25b3d704ffd8f1d4f34e0ff8bd9 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:17.978525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984d05d-74e1-4251-b006-4541f425c7b8 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.978807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984d05d-74e1-4251-b006-4541f425c7b8 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.979104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984d05d-74e1-4251-b006-4541f425c7b8 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.981639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 140/282] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:17.987042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1896-ebfc-4bd6-a0f2-43caf3850d2e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.987395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1896-ebfc-4bd6-a0f2-43caf3850d2e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.987745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1896-ebfc-4bd6-a0f2-43caf3850d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.994343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:17.995974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:17.996428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:17.997072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:17.998729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:17.999237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:18.000820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:18.008480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cb2f1896-ebfc-4bd6-a0f2-43caf3850d2e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:18.009764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cb2f1896-ebfc-4bd6-a0f2-43caf3850d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:18.010839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 143/283] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:18.012132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:18.085824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 141/284] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:18.091650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f1896-ebfc-4bd6-a0f2-43caf3850d2e None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:18.109987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:18.138739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62632c-6f42-4ed8-b81f-38695a65b8b4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:18.208770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:18.209688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:18.210137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:18.210668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:18.211702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] RBAC: Adding query filter params (role.id=95375b0480bf4855a3ce51d1985c9b57, user.id=0bd1f25b3d704ffd8f1d4f34e0ff8bd9, scope.system=all) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:18.212181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:18.214330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:18.215610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f1896-ebfc-4bd6-a0f2-43caf3850d2e None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:18.227896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 144/285] 10.222.0.94 () {56 vars in 1308 bytes} [Fri Aug 26 15:35:18 2022] GET /identity/v3/role_assignments?user.id=0bd1f25b3d704ffd8f1d4f34e0ff8bd9&scope.system=all&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:18.248991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:18.258599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c0b23-9aca-4970-a447-1eb7b1c1b6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:18.259288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c0b23-9aca-4970-a447-1eb7b1c1b6e0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:18.259564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c0b23-9aca-4970-a447-1eb7b1c1b6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:18.259836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c0b23-9aca-4970-a447-1eb7b1c1b6e0 None admin] PATH_INFO: `/v3/users/3aadc0e890a24843a87870a13a9ca70e` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:18.261604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c0b23-9aca-4970-a447-1eb7b1c1b6e0 None admin] RBAC: Authorizing `identity:get_user(user_id=3aadc0e890a24843a87870a13a9ca70e)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:18.263224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c0b23-9aca-4970-a447-1eb7b1c1b6e0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:18.266577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 142/286] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:18 2022] GET /identity/v3/users/3aadc0e890a24843a87870a13a9ca70e => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:18.270648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:18.425890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee4d76-c656-4b47-9480-0de313734f4f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:18.461257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01c23c-e4d5-46b4-a249-2b3395c3dd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:18.461824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c23c-e4d5-46b4-a249-2b3395c3dd24 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:18.462099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c23c-e4d5-46b4-a249-2b3395c3dd24 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:18.462397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c23c-e4d5-46b4-a249-2b3395c3dd24 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:18.463585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01c23c-e4d5-46b4-a249-2b3395c3dd24 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:18.465823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01c23c-e4d5-46b4-a249-2b3395c3dd24 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:18.467812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 145/287] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:18 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:18.474671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c0b23-9aca-4970-a447-1eb7b1c1b6e0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:18.496456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01c23c-e4d5-46b4-a249-2b3395c3dd24 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:18.521988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:18.593553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c0b23-9aca-4970-a447-1eb7b1c1b6e0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:18.647610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01c23c-e4d5-46b4-a249-2b3395c3dd24 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:18.660974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730ec78-084a-4789-bf42-2c741dd4b902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:18.661991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730ec78-084a-4789-bf42-2c741dd4b902 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:18.662441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730ec78-084a-4789-bf42-2c741dd4b902 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:18.662905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730ec78-084a-4789-bf42-2c741dd4b902 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300/users/3aadc0e890a24843a87870a13a9ca70e/roles/3320a9786ec948aab8c8fcbb156eb9fa` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:18.668688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730ec78-084a-4789-bf42-2c741dd4b902 None admin] RBAC: Authorizing `identity:create_grant(project_id=f618dc420f704f2c8a85cfe182a07300, user_id=3aadc0e890a24843a87870a13a9ca70e, role_id=3320a9786ec948aab8c8fcbb156eb9fa)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:18.670463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730ec78-084a-4789-bf42-2c741dd4b902 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:18.701560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e7ec-249d-4f36-bcaf-af6f944ef360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:18.702500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e7ec-249d-4f36-bcaf-af6f944ef360 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:18.702988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e7ec-249d-4f36-bcaf-af6f944ef360 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:18.703455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e7ec-249d-4f36-bcaf-af6f944ef360 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:18.718088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6e7ec-249d-4f36-bcaf-af6f944ef360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ddd6e7ec-249d-4f36-bcaf-af6f944ef360 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:18.723412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:18.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:18.725080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:18.727394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 146/288] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:18.738758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e7ec-249d-4f36-bcaf-af6f944ef360 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:18.794365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 143/289] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:35:18 2022] PUT /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300/users/3aadc0e890a24843a87870a13a9ca70e/roles/3320a9786ec948aab8c8fcbb156eb9fa => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:18.844825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:18.877449 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e7ec-249d-4f36-bcaf-af6f944ef360 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:18.919459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:18.920088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:18.920365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:18.920635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:18.921606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:18.921910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:18.923407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:18.933666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 147/290] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:18.939372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730ec78-084a-4789-bf42-2c741dd4b902 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.004284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:19.056218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730ec78-084a-4789-bf42-2c741dd4b902 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.094947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff598-2872-4703-9223-1aff74cdf93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.095611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff598-2872-4703-9223-1aff74cdf93e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.095886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff598-2872-4703-9223-1aff74cdf93e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.096151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff598-2872-4703-9223-1aff74cdf93e None admin] PATH_INFO: `/v3/users/2d43bf0e9493437e85fd8f8e636ed940` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.098678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fff598-2872-4703-9223-1aff74cdf93e None admin] RBAC: Authorizing `identity:get_user(user_id=2d43bf0e9493437e85fd8f8e636ed940)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:19.100246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fff598-2872-4703-9223-1aff74cdf93e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:19.102168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 144/291] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:18 2022] GET /identity/v3/users/2d43bf0e9493437e85fd8f8e636ed940 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:19.108036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.226340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc9b8e-5dbd-4893-8bdd-501f709b6675 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.278626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fe60-d17c-4bad-a125-052e3b35c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.279186 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fe60-d17c-4bad-a125-052e3b35c590 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.279466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fe60-d17c-4bad-a125-052e3b35c590 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.279739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8fe60-d17c-4bad-a125-052e3b35c590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.293782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:19.376518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8fe60-d17c-4bad-a125-052e3b35c590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:19.377981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8fe60-d17c-4bad-a125-052e3b35c590 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:19.499846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 148/292] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:19.505243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff598-2872-4703-9223-1aff74cdf93e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.616088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff598-2872-4703-9223-1aff74cdf93e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.641907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:19.654759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a29a6-e113-4e52-8e73-cad5924eb99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.660542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a29a6-e113-4e52-8e73-cad5924eb99a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.660542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a29a6-e113-4e52-8e73-cad5924eb99a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.660542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a29a6-e113-4e52-8e73-cad5924eb99a None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.660542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a29a6-e113-4e52-8e73-cad5924eb99a None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:19.660542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a29a6-e113-4e52-8e73-cad5924eb99a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:19.660861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 145/293] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:19.667609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fe60-d17c-4bad-a125-052e3b35c590 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:19.714243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:19.783820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8fe60-d17c-4bad-a125-052e3b35c590 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:19.820857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e95b7c-76a7-4c4f-9cc2-f99ced4902ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:19.821530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e95b7c-76a7-4c4f-9cc2-f99ced4902ae None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:19.821823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e95b7c-76a7-4c4f-9cc2-f99ced4902ae None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:19.822121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e95b7c-76a7-4c4f-9cc2-f99ced4902ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:19.822859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e95b7c-76a7-4c4f-9cc2-f99ced4902ae None admin] RBAC: Adding query filter params (role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=f618dc420f704f2c8a85cfe182a07300, user.id=2d43bf0e9493437e85fd8f8e636ed940) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:19.823830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e95b7c-76a7-4c4f-9cc2-f99ced4902ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:19.825483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e95b7c-76a7-4c4f-9cc2-f99ced4902ae None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:19.832372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 149/294] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:19 2022] GET /identity/v3/role_assignments?user.id=2d43bf0e9493437e85fd8f8e636ed940&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:20.143021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675661-513f-464a-885d-2bd21b2468b0 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.143435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675661-513f-464a-885d-2bd21b2468b0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.143607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10675661-513f-464a-885d-2bd21b2468b0 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.145365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 146/295] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:20.154066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2f336-d684-40b9-93fc-1cb926e07e92 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.154357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2f336-d684-40b9-93fc-1cb926e07e92 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.154618 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2f336-d684-40b9-93fc-1cb926e07e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.188071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-cff2f336-d684-40b9-93fc-1cb926e07e92 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:20.189317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-cff2f336-d684-40b9-93fc-1cb926e07e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:20.292503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaa78-325a-4f81-880e-662f9c384665 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.292930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaa78-325a-4f81-880e-662f9c384665 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.293144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaa78-325a-4f81-880e-662f9c384665 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.294861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 147/296] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:20.300700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4634460-adfb-41e7-9c59-ce0d301da20f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.301302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4634460-adfb-41e7-9c59-ce0d301da20f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.301834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4634460-adfb-41e7-9c59-ce0d301da20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.314333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 150/297] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:20.324129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e4634460-adfb-41e7-9c59-ce0d301da20f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:20.325735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e4634460-adfb-41e7-9c59-ce0d301da20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:20.405669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 148/298] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:20.685586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2f336-d684-40b9-93fc-1cb926e07e92 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:20.702786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46105c5-44d4-4e15-84fc-c9feaa96299c None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.703359 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46105c5-44d4-4e15-84fc-c9feaa96299c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.703359 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46105c5-44d4-4e15-84fc-c9feaa96299c None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.704946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 149/299] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:20.709821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe6d0a-c567-471a-9938-46991f49f9ee None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.710099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe6d0a-c567-471a-9938-46991f49f9ee None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.710369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe6d0a-c567-471a-9938-46991f49f9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.728504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:20.732727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-97fe6d0a-c567-471a-9938-46991f49f9ee None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:20.734713 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-97fe6d0a-c567-471a-9938-46991f49f9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:20.813603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 150/300] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:20.815887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe6d0a-c567-471a-9938-46991f49f9ee None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:20.827306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2f336-d684-40b9-93fc-1cb926e07e92 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:20.880745 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d57ef-f4fb-40cf-85c4-fdeca0e217c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:20.881759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d57ef-f4fb-40cf-85c4-fdeca0e217c9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:20.882102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d57ef-f4fb-40cf-85c4-fdeca0e217c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:20.882492 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d57ef-f4fb-40cf-85c4-fdeca0e217c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:20.884075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d57ef-f4fb-40cf-85c4-fdeca0e217c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:20.885116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:20.888135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d57ef-f4fb-40cf-85c4-fdeca0e217c9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:20.890754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 151/301] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:20.896278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d57ef-f4fb-40cf-85c4-fdeca0e217c9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:20.956645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe6d0a-c567-471a-9938-46991f49f9ee None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:20.998974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76701a86-959e-46ab-b71a-4457a782e57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:20.999993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76701a86-959e-46ab-b71a-4457a782e57b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.000483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76701a86-959e-46ab-b71a-4457a782e57b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.001056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76701a86-959e-46ab-b71a-4457a782e57b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.013680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76701a86-959e-46ab-b71a-4457a782e57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-76701a86-959e-46ab-b71a-4457a782e57b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:21.019256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:21.020099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:21.020850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:21.023302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 151/302] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:21.027682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:21.030668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76701a86-959e-46ab-b71a-4457a782e57b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.032431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d57ef-f4fb-40cf-85c4-fdeca0e217c9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.076525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74080b5-6c2e-43a5-8681-aadb27427fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.077542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74080b5-6c2e-43a5-8681-aadb27427fa1 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.078006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74080b5-6c2e-43a5-8681-aadb27427fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.078534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74080b5-6c2e-43a5-8681-aadb27427fa1 None admin] PATH_INFO: `/v3/groups` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.079421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74080b5-6c2e-43a5-8681-aadb27427fa1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.081185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74080b5-6c2e-43a5-8681-aadb27427fa1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.145789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76701a86-959e-46ab-b71a-4457a782e57b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.157492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:21.160553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 152/303] 10.222.0.94 () {60 vars in 1134 bytes} [Fri Aug 26 15:35:20 2022] POST /identity/v3/groups => generated 229 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:35:21.169457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74080b5-6c2e-43a5-8681-aadb27427fa1 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.186371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd8639-71e8-4482-80a4-08ad8c984c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.187097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd8639-71e8-4482-80a4-08ad8c984c49 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.187383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd8639-71e8-4482-80a4-08ad8c984c49 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.187648 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd8639-71e8-4482-80a4-08ad8c984c49 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.188339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd8639-71e8-4482-80a4-08ad8c984c49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:21.188689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd8639-71e8-4482-80a4-08ad8c984c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.190121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd8639-71e8-4482-80a4-08ad8c984c49 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.201779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 152/304] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:21.203449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763a3e4-8012-48bf-8d57-67a2e5c4aea0 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.203738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763a3e4-8012-48bf-8d57-67a2e5c4aea0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.204321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763a3e4-8012-48bf-8d57-67a2e5c4aea0 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.205368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 153/305] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:21.208366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763a3e4-8012-48bf-8d57-67a2e5c4aea0 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.305919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74080b5-6c2e-43a5-8681-aadb27427fa1 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.338136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763a3e4-8012-48bf-8d57-67a2e5c4aea0 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.346132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e7f18-300a-4bfc-8111-22462cfe7c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.347441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7f18-300a-4bfc-8111-22462cfe7c4c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.347441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7f18-300a-4bfc-8111-22462cfe7c4c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.347441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e7f18-300a-4bfc-8111-22462cfe7c4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.359479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e7f18-300a-4bfc-8111-22462cfe7c4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-1f6e7f18-300a-4bfc-8111-22462cfe7c4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:21.364742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:21.365796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:21.366801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 153/306] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:21.372882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4799a-5f31-4239-966d-049e21b23d86 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.373171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4799a-5f31-4239-966d-049e21b23d86 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.373446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4799a-5f31-4239-966d-049e21b23d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.381984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49a3f0-9f76-493d-9026-e5e358cfc2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.382827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49a3f0-9f76-493d-9026-e5e358cfc2a3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.383261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49a3f0-9f76-493d-9026-e5e358cfc2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.383688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49a3f0-9f76-493d-9026-e5e358cfc2a3 None admin] PATH_INFO: `/v3/users/df6ddca4041849ad8ea733e5fdc1020f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.392147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ebc4799a-5f31-4239-966d-049e21b23d86 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:21.393330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ebc4799a-5f31-4239-966d-049e21b23d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:21.399227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49a3f0-9f76-493d-9026-e5e358cfc2a3 None admin] RBAC: Authorizing `identity:get_user(user_id=df6ddca4041849ad8ea733e5fdc1020f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.401774 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49a3f0-9f76-493d-9026-e5e358cfc2a3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.408980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 154/307] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:21 2022] GET /identity/v3/users/df6ddca4041849ad8ea733e5fdc1020f => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:21.408980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49a3f0-9f76-493d-9026-e5e358cfc2a3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.409882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:21.444886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:21.528258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49a3f0-9f76-493d-9026-e5e358cfc2a3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.564434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.565047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.565341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.565653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.566276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] RBAC: Adding query filter params (name=member) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:21.566576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.567974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.579242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 154/308] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:21.580388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4799a-5f31-4239-966d-049e21b23d86 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.584514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 155/309] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:21.593464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.696403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4799a-5f31-4239-966d-049e21b23d86 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.705041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:21.716764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24b8dd-4768-49c8-9cb5-0a38a758840c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.741358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.742582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.743075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.743538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.744772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] RBAC: Adding query filter params (role.id=3320a9786ec948aab8c8fcbb156eb9fa, user.id=df6ddca4041849ad8ea733e5fdc1020f, scope.system=all) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:21.745943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.748011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.754395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 155/310] 10.222.0.94 () {56 vars in 1308 bytes} [Fri Aug 26 15:35:21 2022] GET /identity/v3/role_assignments?user.id=df6ddca4041849ad8ea733e5fdc1020f&scope.system=all&role.id=3320a9786ec948aab8c8fcbb156eb9fa => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:21.760661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2ad8-99f5-4d94-8443-f8f59fd98b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.761331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2ad8-99f5-4d94-8443-f8f59fd98b25 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.761607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2ad8-99f5-4d94-8443-f8f59fd98b25 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.761876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2ad8-99f5-4d94-8443-f8f59fd98b25 None admin] PATH_INFO: `/v3/users/0bd1f25b3d704ffd8f1d4f34e0ff8bd9` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.767307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f2ad8-99f5-4d94-8443-f8f59fd98b25 None admin] RBAC: Authorizing `identity:get_user(user_id=0bd1f25b3d704ffd8f1d4f34e0ff8bd9)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.767307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f2ad8-99f5-4d94-8443-f8f59fd98b25 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.767307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 156/311] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:21 2022] GET /identity/v3/users/0bd1f25b3d704ffd8f1d4f34e0ff8bd9 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:21.776866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.797045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:21.856336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2ad8-99f5-4d94-8443-f8f59fd98b25 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:21.904611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6464b-4bab-4950-b74b-22dd61544bd3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:21.940662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea32fc-b127-454d-8dcc-99d03f400c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:21.941331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea32fc-b127-454d-8dcc-99d03f400c92 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:21.941609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea32fc-b127-454d-8dcc-99d03f400c92 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:21.941881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea32fc-b127-454d-8dcc-99d03f400c92 None admin] PATH_INFO: `/v3/system/users/0bd1f25b3d704ffd8f1d4f34e0ff8bd9/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:21.943343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea32fc-b127-454d-8dcc-99d03f400c92 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0bd1f25b3d704ffd8f1d4f34e0ff8bd9, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:21.944734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea32fc-b127-454d-8dcc-99d03f400c92 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:21.969893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2ad8-99f5-4d94-8443-f8f59fd98b25 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:22.014039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb390bce-0126-4563-a1f7-32f5eba77234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:22.014765 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb390bce-0126-4563-a1f7-32f5eba77234 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:22.015074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb390bce-0126-4563-a1f7-32f5eba77234 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:22.015368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb390bce-0126-4563-a1f7-32f5eba77234 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:22.029719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb390bce-0126-4563-a1f7-32f5eba77234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-fb390bce-0126-4563-a1f7-32f5eba77234 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:22.038676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:22.039872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:22.040677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:22.041960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 156/313] 10.222.0.94 () {58 vars in 1334 bytes} [Fri Aug 26 15:35:21 2022] PUT /identity/v3/system/users/0bd1f25b3d704ffd8f1d4f34e0ff8bd9/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:22.046652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 157/313] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:22.048511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea32fc-b127-454d-8dcc-99d03f400c92 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:22.147548 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:22.176208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea32fc-b127-454d-8dcc-99d03f400c92 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:22.219388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:22.220006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:22.220341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:22.220579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:22.221274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] RBAC: Adding query filter params (name=reader) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:22.221551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:22.222953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:22.233600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 157/314] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:22.239265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb390bce-0126-4563-a1f7-32f5eba77234 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:22.267509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:22.352903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb390bce-0126-4563-a1f7-32f5eba77234 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:22.395519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305bd05b-880e-492e-b434-dcf963a26ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:22.396169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bd05b-880e-492e-b434-dcf963a26ddb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:22.397325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bd05b-880e-492e-b434-dcf963a26ddb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:22.397601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bd05b-880e-492e-b434-dcf963a26ddb None admin] PATH_INFO: `/v3/users/3aadc0e890a24843a87870a13a9ca70e` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:22.398790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305bd05b-880e-492e-b434-dcf963a26ddb None admin] RBAC: Authorizing `identity:get_user(user_id=3aadc0e890a24843a87870a13a9ca70e)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:22.403138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305bd05b-880e-492e-b434-dcf963a26ddb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:22.404994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 158/315] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:22 2022] GET /identity/v3/users/3aadc0e890a24843a87870a13a9ca70e => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:22.413153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:22.573786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06d947-0b25-49a1-88b6-c17f5f56319a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:22.611237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84712-71a5-4faa-9eb1-b84f0f7641cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:22.611861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84712-71a5-4faa-9eb1-b84f0f7641cc None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:22.612140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84712-71a5-4faa-9eb1-b84f0f7641cc None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:22.612407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84712-71a5-4faa-9eb1-b84f0f7641cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:22.624025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:22.685784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb84712-71a5-4faa-9eb1-b84f0f7641cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:22.687290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb84712-71a5-4faa-9eb1-b84f0f7641cc None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:22.793730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 158/316] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:22.799380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305bd05b-880e-492e-b434-dcf963a26ddb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:22.898129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:22.912309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305bd05b-880e-492e-b434-dcf963a26ddb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:22.950343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2e5be-185a-4e95-922d-33a39116130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:22.951287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2e5be-185a-4e95-922d-33a39116130d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:22.951880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2e5be-185a-4e95-922d-33a39116130d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:22.952347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2e5be-185a-4e95-922d-33a39116130d None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:22.953635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2e5be-185a-4e95-922d-33a39116130d None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:22.955346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2e5be-185a-4e95-922d-33a39116130d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:22.957440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 159/317] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:22 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:22.963686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84712-71a5-4faa-9eb1-b84f0f7641cc None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:23.008750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:23.072709 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84712-71a5-4faa-9eb1-b84f0f7641cc None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:23.112032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b94a71-717a-4539-a575-e4884f38821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:23.112695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b94a71-717a-4539-a575-e4884f38821b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.112976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b94a71-717a-4539-a575-e4884f38821b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.113245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b94a71-717a-4539-a575-e4884f38821b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.113991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b94a71-717a-4539-a575-e4884f38821b None admin] RBAC: Adding query filter params (role.id=3320a9786ec948aab8c8fcbb156eb9fa, scope.project.id=f618dc420f704f2c8a85cfe182a07300, user.id=3aadc0e890a24843a87870a13a9ca70e) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:23.114291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b94a71-717a-4539-a575-e4884f38821b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:23.115802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b94a71-717a-4539-a575-e4884f38821b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:23.123201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 159/318] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:22 2022] GET /identity/v3/role_assignments?user.id=3aadc0e890a24843a87870a13a9ca70e&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=3320a9786ec948aab8c8fcbb156eb9fa => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:23.232381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9673-120a-4ccd-a4a8-84d1204ca87d None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.232640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9673-120a-4ccd-a4a8-84d1204ca87d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.232884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9673-120a-4ccd-a4a8-84d1204ca87d None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.234319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 160/319] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:23.240317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebab7-01e4-4f2b-a96f-966ac8337e03 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.240750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebab7-01e4-4f2b-a96f-966ac8337e03 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.241113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebab7-01e4-4f2b-a96f-966ac8337e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.262241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3f2ebab7-01e4-4f2b-a96f-966ac8337e03 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:23.263505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3f2ebab7-01e4-4f2b-a96f-966ac8337e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:23.345694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 160/320] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:23.616628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de9673-120a-4ccd-a4a8-84d1204ca87d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:23.710422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf80b3-62f7-4d3e-8226-a8123ec76a2b None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.710741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf80b3-62f7-4d3e-8226-a8123ec76a2b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.710986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf80b3-62f7-4d3e-8226-a8123ec76a2b None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.712469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 161/321] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:23.717276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d2e3-16ed-4998-8b50-4546b748bb1d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.717556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d2e3-16ed-4998-8b50-4546b748bb1d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.717830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d2e3-16ed-4998-8b50-4546b748bb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.727427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de9673-120a-4ccd-a4a8-84d1204ca87d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:23.736758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ebf1d2e3-16ed-4998-8b50-4546b748bb1d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:23.737945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ebf1d2e3-16ed-4998-8b50-4546b748bb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:23.764494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8b9b7-84be-4478-a8db-690b2f9ffca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:23.765273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8b9b7-84be-4478-a8db-690b2f9ffca9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.765555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8b9b7-84be-4478-a8db-690b2f9ffca9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.765862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8b9b7-84be-4478-a8db-690b2f9ffca9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.766413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:23.767705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8b9b7-84be-4478-a8db-690b2f9ffca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:23.769217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8b9b7-84be-4478-a8db-690b2f9ffca9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:23.775093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 161/322] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:23.776162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8b9b7-84be-4478-a8db-690b2f9ffca9 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:23.803048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:23.812813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 162/323] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:23.881475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8b9b7-84be-4478-a8db-690b2f9ffca9 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:23.927080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecb447-537a-462d-ade6-fc5e0ecb7c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:23.927703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb447-537a-462d-ade6-fc5e0ecb7c70 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:23.927978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb447-537a-462d-ade6-fc5e0ecb7c70 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:23.928241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb447-537a-462d-ade6-fc5e0ecb7c70 None admin] PATH_INFO: `/v3/groups` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:23.928871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecb447-537a-462d-ade6-fc5e0ecb7c70 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:23.930337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecb447-537a-462d-ade6-fc5e0ecb7c70 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:24.052865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 162/324] 10.222.0.94 () {60 vars in 1134 bytes} [Fri Aug 26 15:35:23 2022] POST /identity/v3/groups => generated 226 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:35:24.088016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1d2e3-16ed-4998-8b50-4546b748bb1d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:24.176516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8084-3390-454c-82ea-c7555b4e2cf3 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.176686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8084-3390-454c-82ea-c7555b4e2cf3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.176954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8084-3390-454c-82ea-c7555b4e2cf3 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.178464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 163/325] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:24.184441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab6036-8361-49f0-8017-93d56968bc60 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.184736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab6036-8361-49f0-8017-93d56968bc60 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.185077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab6036-8361-49f0-8017-93d56968bc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.202516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1d2e3-16ed-4998-8b50-4546b748bb1d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:24.204405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-0fab6036-8361-49f0-8017-93d56968bc60 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:24.205614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0fab6036-8361-49f0-8017-93d56968bc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:24.243103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3f60-1d07-46f3-9a72-55caaf8a8d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:24.243723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3f60-1d07-46f3-9a72-55caaf8a8d2a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.244003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3f60-1d07-46f3-9a72-55caaf8a8d2a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.244276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3f60-1d07-46f3-9a72-55caaf8a8d2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.257165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c3f60-1d07-46f3-9a72-55caaf8a8d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-338c3f60-1d07-46f3-9a72-55caaf8a8d2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:24.261998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:24.263024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:24.263843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 163/326] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:24.270170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3f60-1d07-46f3-9a72-55caaf8a8d2a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:24.291563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 164/327] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:24.302092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:24.384713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3f60-1d07-46f3-9a72-55caaf8a8d2a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:24.424508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:24.425126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.425398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.425665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.426334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] RBAC: Adding query filter params (name=reader) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:24.426681 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:24.428174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:24.437916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 164/328] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:24.443645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab6036-8361-49f0-8017-93d56968bc60 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:24.461411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:24.556573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab6036-8361-49f0-8017-93d56968bc60 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:24.573462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:24.600664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edb71f-a57b-42aa-a436-5cc5934f7349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:24.601310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edb71f-a57b-42aa-a436-5cc5934f7349 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.601583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edb71f-a57b-42aa-a436-5cc5934f7349 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.601849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edb71f-a57b-42aa-a436-5cc5934f7349 None admin] PATH_INFO: `/v3/users/df6ddca4041849ad8ea733e5fdc1020f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.606565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edb71f-a57b-42aa-a436-5cc5934f7349 None admin] RBAC: Authorizing `identity:get_user(user_id=df6ddca4041849ad8ea733e5fdc1020f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:24.607876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edb71f-a57b-42aa-a436-5cc5934f7349 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:24.609834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 165/329] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:24 2022] GET /identity/v3/users/df6ddca4041849ad8ea733e5fdc1020f => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:24.615337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edb71f-a57b-42aa-a436-5cc5934f7349 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:24.680005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:24.699261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0ffbc-887f-43be-802c-bb029c066ada None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:24.715443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:24.738659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edb71f-a57b-42aa-a436-5cc5934f7349 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:24.748961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1954580-871b-4f0a-a26d-d25c8e33dca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:24.750293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1954580-871b-4f0a-a26d-d25c8e33dca0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.750668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1954580-871b-4f0a-a26d-d25c8e33dca0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.751074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1954580-871b-4f0a-a26d-d25c8e33dca0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.767928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1954580-871b-4f0a-a26d-d25c8e33dca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-d1954580-871b-4f0a-a26d-d25c8e33dca0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:24.777149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:24.778222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:24.779074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 165/330] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:24.784060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1954580-871b-4f0a-a26d-d25c8e33dca0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:24.795697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893dfe3-138f-49e7-9a26-8b3ff823ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:24.796383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893dfe3-138f-49e7-9a26-8b3ff823ad14 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.796661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893dfe3-138f-49e7-9a26-8b3ff823ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.796934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893dfe3-138f-49e7-9a26-8b3ff823ad14 None admin] PATH_INFO: `/v3/system/users/df6ddca4041849ad8ea733e5fdc1020f/roles/3320a9786ec948aab8c8fcbb156eb9fa` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.798444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893dfe3-138f-49e7-9a26-8b3ff823ad14 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=df6ddca4041849ad8ea733e5fdc1020f, role_id=3320a9786ec948aab8c8fcbb156eb9fa)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:24.802192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893dfe3-138f-49e7-9a26-8b3ff823ad14 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:24.879564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 166/331] 10.222.0.94 () {58 vars in 1334 bytes} [Fri Aug 26 15:35:24 2022] PUT /identity/v3/system/users/df6ddca4041849ad8ea733e5fdc1020f/roles/3320a9786ec948aab8c8fcbb156eb9fa => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:35:24.911586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1954580-871b-4f0a-a26d-d25c8e33dca0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:24.956038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:24.956827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:24.957098 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:24.957373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:24.958065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:24.958367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:24.959805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:24.972474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 166/332] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:24.977947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893dfe3-138f-49e7-9a26-8b3ff823ad14 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:25.102139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893dfe3-138f-49e7-9a26-8b3ff823ad14 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:25.143251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc40c41-9f9c-40ec-9f0c-aa59ccb72399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:25.143894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40c41-9f9c-40ec-9f0c-aa59ccb72399 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:25.144171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40c41-9f9c-40ec-9f0c-aa59ccb72399 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:25.144440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc40c41-9f9c-40ec-9f0c-aa59ccb72399 None admin] PATH_INFO: `/v3/users/0bd1f25b3d704ffd8f1d4f34e0ff8bd9` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:25.145990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc40c41-9f9c-40ec-9f0c-aa59ccb72399 None admin] RBAC: Authorizing `identity:get_user(user_id=0bd1f25b3d704ffd8f1d4f34e0ff8bd9)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:25.147486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc40c41-9f9c-40ec-9f0c-aa59ccb72399 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:25.149197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 167/333] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:24 2022] GET /identity/v3/users/0bd1f25b3d704ffd8f1d4f34e0ff8bd9 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:25.156942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:25.169451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:25.270232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a82da-3e1c-4beb-9596-078f1b990342 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:25.310530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fe0ad-e360-4db8-b66e-10eb3178d193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:25.311176 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fe0ad-e360-4db8-b66e-10eb3178d193 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:25.311456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fe0ad-e360-4db8-b66e-10eb3178d193 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:25.311728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fe0ad-e360-4db8-b66e-10eb3178d193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:25.312464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fe0ad-e360-4db8-b66e-10eb3178d193 None admin] RBAC: Adding query filter params (role.id=95375b0480bf4855a3ce51d1985c9b57, user.id=0bd1f25b3d704ffd8f1d4f34e0ff8bd9, scope.system=all) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:25.312772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fe0ad-e360-4db8-b66e-10eb3178d193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:25.314209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fe0ad-e360-4db8-b66e-10eb3178d193 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:25.327764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 167/334] 10.222.0.94 () {56 vars in 1308 bytes} [Fri Aug 26 15:35:25 2022] GET /identity/v3/role_assignments?user.id=0bd1f25b3d704ffd8f1d4f34e0ff8bd9&scope.system=all&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:26.306396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee73cc3-9a40-490b-9d29-4a21bfdee239 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:26.306905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee73cc3-9a40-490b-9d29-4a21bfdee239 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:26.306905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee73cc3-9a40-490b-9d29-4a21bfdee239 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:26.309286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 168/335] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:26.315932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d088-2fb2-4a2f-9d63-690596a74089 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:26.316399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d088-2fb2-4a2f-9d63-690596a74089 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:26.316472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9d088-2fb2-4a2f-9d63-690596a74089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:26.339414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-9aa9d088-2fb2-4a2f-9d63-690596a74089 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:26.340591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9aa9d088-2fb2-4a2f-9d63-690596a74089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:26.419232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:26.433747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 168/336] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:26.741603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee73cc3-9a40-490b-9d29-4a21bfdee239 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:26.763225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:26.862850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee73cc3-9a40-490b-9d29-4a21bfdee239 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:26.903688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b02e4b-1d43-4935-a4af-b3337efb7ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:26.904329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02e4b-1d43-4935-a4af-b3337efb7ae0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:26.904608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02e4b-1d43-4935-a4af-b3337efb7ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:26.904874 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02e4b-1d43-4935-a4af-b3337efb7ae0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:26.927849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b02e4b-1d43-4935-a4af-b3337efb7ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-b7b02e4b-1d43-4935-a4af-b3337efb7ae0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:26.933379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:26.934505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:26.935341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 169/337] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:26.941958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9d088-2fb2-4a2f-9d63-690596a74089 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:27.068671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e651d5e-ee82-4167-9501-364f61f19b61 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:27.069518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e651d5e-ee82-4167-9501-364f61f19b61 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:27.069801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e651d5e-ee82-4167-9501-364f61f19b61 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:27.071410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 170/338] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:27.077242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6778170-8159-42cf-a5be-9719695490a9 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:27.077522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6778170-8159-42cf-a5be-9719695490a9 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:27.077785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6778170-8159-42cf-a5be-9719695490a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:27.084857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9d088-2fb2-4a2f-9d63-690596a74089 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:27.118008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b6778170-8159-42cf-a5be-9719695490a9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:27.119140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b6778170-8159-42cf-a5be-9719695490a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:27.161680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:27.162303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:27.162573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:27.162838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:27.163535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:27.163829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:27.165212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:27.193616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 169/339] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:27.199731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:27.298535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 171/340] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:27.318946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:27.381224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4fc61-a9e1-47d7-b27d-4e649cbf3014 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:27.423604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffcd7f-86d0-478f-8be4-91d509dc6f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:27.424231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffcd7f-86d0-478f-8be4-91d509dc6f74 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:27.424506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffcd7f-86d0-478f-8be4-91d509dc6f74 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:27.424778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffcd7f-86d0-478f-8be4-91d509dc6f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:27.508570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffcd7f-86d0-478f-8be4-91d509dc6f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:27.510009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffcd7f-86d0-478f-8be4-91d509dc6f74 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:27.597788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:27.655595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 170/341] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:27.660766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6778170-8159-42cf-a5be-9719695490a9 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:27.665198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:27.750549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffcd7f-86d0-478f-8be4-91d509dc6f74 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:27.804453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6778170-8159-42cf-a5be-9719695490a9 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:27.873123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ffc7f-809c-44a3-8df7-7260cd7b71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:27.873768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ffc7f-809c-44a3-8df7-7260cd7b71cb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:27.874039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ffc7f-809c-44a3-8df7-7260cd7b71cb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:27.874300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ffc7f-809c-44a3-8df7-7260cd7b71cb None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:27.875568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ffc7f-809c-44a3-8df7-7260cd7b71cb None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:27.876998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ffc7f-809c-44a3-8df7-7260cd7b71cb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:27.878646 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 172/342] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:27 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:27.884150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfada91-d159-430e-ae0f-206651eb5ef0 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:27.884423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfada91-d159-430e-ae0f-206651eb5ef0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:27.884688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfada91-d159-430e-ae0f-206651eb5ef0 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:27.886637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 173/343] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:27.891275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfada91-d159-430e-ae0f-206651eb5ef0 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:27.950420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffcd7f-86d0-478f-8be4-91d509dc6f74 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:28.009403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09605dab-9454-45ca-b18e-373dd59845d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:28.010042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09605dab-9454-45ca-b18e-373dd59845d8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.010338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09605dab-9454-45ca-b18e-373dd59845d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.010610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09605dab-9454-45ca-b18e-373dd59845d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.018096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:28.023167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09605dab-9454-45ca-b18e-373dd59845d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-09605dab-9454-45ca-b18e-373dd59845d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:28.029593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:28.030453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:28.031269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 171/344] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:28.036953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b18c-35c4-4246-9be9-6903513ba766 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.037236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b18c-35c4-4246-9be9-6903513ba766 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.037508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2b18c-35c4-4246-9be9-6903513ba766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.060312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e6f2b18c-35c4-4246-9be9-6903513ba766 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:28.061613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e6f2b18c-35c4-4246-9be9-6903513ba766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:28.062887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfada91-d159-430e-ae0f-206651eb5ef0 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:28.080217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:28.115244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19c2e4-a243-44de-8f9c-10675ed2d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:28.115895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19c2e4-a243-44de-8f9c-10675ed2d864 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.116165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19c2e4-a243-44de-8f9c-10675ed2d864 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.116430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19c2e4-a243-44de-8f9c-10675ed2d864 None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.125485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19c2e4-a243-44de-8f9c-10675ed2d864 None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:28.127243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19c2e4-a243-44de-8f9c-10675ed2d864 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:28.129114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 174/345] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:27 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:28.130183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5a6d3-81d1-4da4-949e-d75d0ae95539 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.130510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5a6d3-81d1-4da4-949e-d75d0ae95539 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.130782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5a6d3-81d1-4da4-949e-d75d0ae95539 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.132382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 175/346] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:28.133082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5a6d3-81d1-4da4-949e-d75d0ae95539 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:28.159379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 172/347] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:28.160442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2b18c-35c4-4246-9be9-6903513ba766 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:28.255574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5a6d3-81d1-4da4-949e-d75d0ae95539 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:28.284388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2b18c-35c4-4246-9be9-6903513ba766 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:28.296195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e2dfa-5565-426b-9e5e-dfe10ee196bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:28.297135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e2dfa-5565-426b-9e5e-dfe10ee196bf None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.297660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e2dfa-5565-426b-9e5e-dfe10ee196bf None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.298115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e2dfa-5565-426b-9e5e-dfe10ee196bf None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.299026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e2dfa-5565-426b-9e5e-dfe10ee196bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:28.299516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e2dfa-5565-426b-9e5e-dfe10ee196bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:28.301211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e2dfa-5565-426b-9e5e-dfe10ee196bf None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:28.308056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:28.312191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 176/348] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:28.319294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8687990-39df-4ef6-b891-9e91667cda15 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.319965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8687990-39df-4ef6-b891-9e91667cda15 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.320424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8687990-39df-4ef6-b891-9e91667cda15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.341807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:28.343822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.345284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.348770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.348770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] RBAC: Adding query filter params (group.id=8c8057e0f50d4775902e859951a8277c, role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:28.348770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:28.350629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:28.362786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b8687990-39df-4ef6-b891-9e91667cda15 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:28.364719 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 173/349] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:28 2022] GET /identity/v3/role_assignments?group.id=8c8057e0f50d4775902e859951a8277c&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:28.366504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:28.367635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b8687990-39df-4ef6-b891-9e91667cda15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:28.402552 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:28.525627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 177/350] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:28.550775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09eafe-c941-46b0-81f9-0bccf10024ff None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:28.596338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd109-b087-46bb-8387-f4b644e02cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:28.596967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd109-b087-46bb-8387-f4b644e02cb2 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.597242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd109-b087-46bb-8387-f4b644e02cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.597512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd109-b087-46bb-8387-f4b644e02cb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.690408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd109-b087-46bb-8387-f4b644e02cb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:28.693907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd109-b087-46bb-8387-f4b644e02cb2 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:28.719299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:28.728263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0e0a8-480c-48c8-8c64-089b257405e2 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.729193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0e0a8-480c-48c8-8c64-089b257405e2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.735026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0e0a8-480c-48c8-8c64-089b257405e2 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.735026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 178/351] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:28.737374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c65f2-cc7b-4e5f-bf6e-ecaccad2e8fd None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:28.737861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c65f2-cc7b-4e5f-bf6e-ecaccad2e8fd None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:28.738326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c65f2-cc7b-4e5f-bf6e-ecaccad2e8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:28.762683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-619c65f2-cc7b-4e5f-bf6e-ecaccad2e8fd None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:28.764326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-619c65f2-cc7b-4e5f-bf6e-ecaccad2e8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:28.820890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 174/352] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:28.821673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd109-b087-46bb-8387-f4b644e02cb2 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:28.887297 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 179/353] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:28.888607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c65f2-cc7b-4e5f-bf6e-ecaccad2e8fd None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:28.967825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd109-b087-46bb-8387-f4b644e02cb2 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:28.975286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:28.986684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.028563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2128b-2325-4549-a92a-e81a29441661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.029217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2128b-2325-4549-a92a-e81a29441661 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.029497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2128b-2325-4549-a92a-e81a29441661 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.029755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2128b-2325-4549-a92a-e81a29441661 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.036142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c65f2-cc7b-4e5f-bf6e-ecaccad2e8fd None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.059588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b2128b-2325-4549-a92a-e81a29441661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-42b2128b-2325-4549-a92a-e81a29441661 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:29.064508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:29.065411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:29.066201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 175/354] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:29.066854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2128b-2325-4549-a92a-e81a29441661 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.084164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981041fa-abd1-45d3-a390-628ed70586a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.084823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981041fa-abd1-45d3-a390-628ed70586a5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.085127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981041fa-abd1-45d3-a390-628ed70586a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.085404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981041fa-abd1-45d3-a390-628ed70586a5 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.089209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981041fa-abd1-45d3-a390-628ed70586a5 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.091016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981041fa-abd1-45d3-a390-628ed70586a5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.092980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 180/355] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:28 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:29.093847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981041fa-abd1-45d3-a390-628ed70586a5 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.183817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2128b-2325-4549-a92a-e81a29441661 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.224597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db9ef9-caeb-4923-9d04-98ee4f84db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.225249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9ef9-caeb-4923-9d04-98ee4f84db5b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.225527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9ef9-caeb-4923-9d04-98ee4f84db5b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.225797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9ef9-caeb-4923-9d04-98ee4f84db5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.254284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db9ef9-caeb-4923-9d04-98ee4f84db5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-78db9ef9-caeb-4923-9d04-98ee4f84db5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:29.260276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:29.261501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:29.262277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:29.262856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 176/356] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:29.265343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981041fa-abd1-45d3-a390-628ed70586a5 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.272359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db9ef9-caeb-4923-9d04-98ee4f84db5b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.319287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.339802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.339802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.339802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.339802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.340165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:29.340165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.340505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.343106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.351961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 181/357] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:29.359165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.446032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db9ef9-caeb-4923-9d04-98ee4f84db5b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.490801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596b2fe-849a-41ea-be27-5728fa4310eb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.493927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2201c-3384-43cc-9a11-ec1c2cb63a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.495354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2201c-3384-43cc-9a11-ec1c2cb63a49 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.495977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2201c-3384-43cc-9a11-ec1c2cb63a49 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.496250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2201c-3384-43cc-9a11-ec1c2cb63a49 None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.497420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2201c-3384-43cc-9a11-ec1c2cb63a49 None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2201c-3384-43cc-9a11-ec1c2cb63a49 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.500489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 177/358] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:29.501240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2201c-3384-43cc-9a11-ec1c2cb63a49 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.540615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76059a0-2217-460f-adb3-164e537b3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.541237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76059a0-2217-460f-adb3-164e537b3b98 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.542483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76059a0-2217-460f-adb3-164e537b3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.542483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76059a0-2217-460f-adb3-164e537b3b98 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.542483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76059a0-2217-460f-adb3-164e537b3b98 None admin] RBAC: Adding query filter params (name=member) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:29.542881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76059a0-2217-460f-adb3-164e537b3b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.544328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76059a0-2217-460f-adb3-164e537b3b98 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.554377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 182/359] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:29.555663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4e26-c779-437e-81cf-148e508976d2 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.555946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4e26-c779-437e-81cf-148e508976d2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.556215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4e26-c779-437e-81cf-148e508976d2 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.557975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 183/360] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:29.558672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d4e26-c779-437e-81cf-148e508976d2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.646265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.647224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.652266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2201c-3384-43cc-9a11-ec1c2cb63a49 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.700874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e118-24a9-447b-add0-a635bd5b74ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.701993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e118-24a9-447b-add0-a635bd5b74ca None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.702460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e118-24a9-447b-add0-a635bd5b74ca None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.702943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702e118-24a9-447b-add0-a635bd5b74ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.722138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702e118-24a9-447b-add0-a635bd5b74ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-4702e118-24a9-447b-add0-a635bd5b74ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:29.727925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:29.728807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:29.730157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:29.732658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 178/361] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:29.735015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d4e26-c779-437e-81cf-148e508976d2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.735475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e118-24a9-447b-add0-a635bd5b74ca None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.785513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0e8c8-ce50-4923-87ca-d701309bc7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.786159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0e8c8-ce50-4923-87ca-d701309bc7a8 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.786443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0e8c8-ce50-4923-87ca-d701309bc7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.786705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0e8c8-ce50-4923-87ca-d701309bc7a8 None admin] PATH_INFO: `/v3/users/df6ddca4041849ad8ea733e5fdc1020f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.788229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0e8c8-ce50-4923-87ca-d701309bc7a8 None admin] RBAC: Authorizing `identity:get_user(user_id=df6ddca4041849ad8ea733e5fdc1020f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.790726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0e8c8-ce50-4923-87ca-d701309bc7a8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.792764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 184/362] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/users/df6ddca4041849ad8ea733e5fdc1020f => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:29.793839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0e8c8-ce50-4923-87ca-d701309bc7a8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:29.882086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702e118-24a9-447b-add0-a635bd5b74ca None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.933522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11124451-ead4-41b8-8a27-14a4d0840687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.934071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11124451-ead4-41b8-8a27-14a4d0840687 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.934352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11124451-ead4-41b8-8a27-14a4d0840687 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.934618 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11124451-ead4-41b8-8a27-14a4d0840687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.935386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11124451-ead4-41b8-8a27-14a4d0840687 None admin] RBAC: Adding query filter params (group.id=8c8057e0f50d4775902e859951a8277c, role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:29.935689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11124451-ead4-41b8-8a27-14a4d0840687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:29.937352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11124451-ead4-41b8-8a27-14a4d0840687 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:29.940086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0e8c8-ce50-4923-87ca-d701309bc7a8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:29.943846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 179/363] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/role_assignments?group.id=8c8057e0f50d4775902e859951a8277c&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:29.945064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae366-698a-4ac6-aba8-83631642ede9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.945348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae366-698a-4ac6-aba8-83631642ede9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.945623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae366-698a-4ac6-aba8-83631642ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:29.966714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:29.975319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-25eae366-698a-4ac6-aba8-83631642ede9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:29.991797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-25eae366-698a-4ac6-aba8-83631642ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:29.992187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3493bd-ff95-4ff0-bb94-3b3d37d75a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:29.992187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3493bd-ff95-4ff0-bb94-3b3d37d75a1a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:29.992187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3493bd-ff95-4ff0-bb94-3b3d37d75a1a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:29.992187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3493bd-ff95-4ff0-bb94-3b3d37d75a1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:30.003208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:30.081220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 180/364] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:30.082555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eae366-698a-4ac6-aba8-83631642ede9 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:30.097213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3493bd-ff95-4ff0-bb94-3b3d37d75a1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:30.099239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3493bd-ff95-4ff0-bb94-3b3d37d75a1a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:30.288963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 185/365] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:30.289569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3493bd-ff95-4ff0-bb94-3b3d37d75a1a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:30.296086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eae366-698a-4ac6-aba8-83631642ede9 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:30.334979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:30.351848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:30.352593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:30.352978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:30.353785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:30.355092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:30.355405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:30.356890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:30.368452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 181/366] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:30.369309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:30.395436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:30.489876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3493bd-ff95-4ff0-bb94-3b3d37d75a1a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:30.503396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76a365-34c5-4011-8b30-62ef47f5ff2d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:30.537243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:30.538108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:30.538445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:30.538786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:30.539675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] RBAC: Adding query filter params (role.id=3320a9786ec948aab8c8fcbb156eb9fa, user.id=df6ddca4041849ad8ea733e5fdc1020f, scope.system=all) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:30.539976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:30.542279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:30.549010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a31f9d-56a1-4869-8b69-235a9eac9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:30.549964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a31f9d-56a1-4869-8b69-235a9eac9929 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:30.550443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a31f9d-56a1-4869-8b69-235a9eac9929 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:30.551075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a31f9d-56a1-4869-8b69-235a9eac9929 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:30.553303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a31f9d-56a1-4869-8b69-235a9eac9929 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:30.555479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a31f9d-56a1-4869-8b69-235a9eac9929 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:30.558547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 182/367] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:30 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:30.559402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 186/368] 10.222.0.94 () {56 vars in 1308 bytes} [Fri Aug 26 15:35:30 2022] GET /identity/v3/role_assignments?user.id=df6ddca4041849ad8ea733e5fdc1020f&scope.system=all&role.id=3320a9786ec948aab8c8fcbb156eb9fa => generated 510 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:30.559942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a31f9d-56a1-4869-8b69-235a9eac9929 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:30.560249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:30.686945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:30.707720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a31f9d-56a1-4869-8b69-235a9eac9929 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:30.741692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:30.753426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303806df-7507-42e6-89a8-a21665cd4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:30.754781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303806df-7507-42e6-89a8-a21665cd4c54 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:30.755104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303806df-7507-42e6-89a8-a21665cd4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:30.755382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303806df-7507-42e6-89a8-a21665cd4c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:30.778287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c78f0d-57a9-4d5e-af9f-07254f659bf0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:30.825027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000826c-6e25-4476-a284-6cbe6473dac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:30.826122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000826c-6e25-4476-a284-6cbe6473dac4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:30.826606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000826c-6e25-4476-a284-6cbe6473dac4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:30.828206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000826c-6e25-4476-a284-6cbe6473dac4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:30.843237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000826c-6e25-4476-a284-6cbe6473dac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:30.849123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303806df-7507-42e6-89a8-a21665cd4c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8000826c-6e25-4476-a284-6cbe6473dac4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:30.849587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:30.850435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:30.851569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:30.852151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303806df-7507-42e6-89a8-a21665cd4c54 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:30.854020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 187/369] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:30.855324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000826c-6e25-4476-a284-6cbe6473dac4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:30.965580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:30.986651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 183/370] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:30.989343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630f467-a85c-49b1-9ec7-7d26e2e5d8f8 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:30.989859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630f467-a85c-49b1-9ec7-7d26e2e5d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:30.990274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630f467-a85c-49b1-9ec7-7d26e2e5d8f8 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:30.992565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 184/371] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:30.993503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630f467-a85c-49b1-9ec7-7d26e2e5d8f8 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:30.999611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000826c-6e25-4476-a284-6cbe6473dac4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.042696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58d9dc-dd61-4161-ad20-7336201a187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.043358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58d9dc-dd61-4161-ad20-7336201a187e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.043631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58d9dc-dd61-4161-ad20-7336201a187e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.043951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58d9dc-dd61-4161-ad20-7336201a187e None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.045055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58d9dc-dd61-4161-ad20-7336201a187e None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:31.046489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58d9dc-dd61-4161-ad20-7336201a187e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:31.053133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 188/372] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:30 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:31.054723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58d9dc-dd61-4161-ad20-7336201a187e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.060699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:31.134288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630f467-a85c-49b1-9ec7-7d26e2e5d8f8 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.174444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8230e-c0cc-486e-9781-966d8633b986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.175093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8230e-c0cc-486e-9781-966d8633b986 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.175823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8230e-c0cc-486e-9781-966d8633b986 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.175823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8230e-c0cc-486e-9781-966d8633b986 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.176296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8230e-c0cc-486e-9781-966d8633b986 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:31.176600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8230e-c0cc-486e-9781-966d8633b986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:31.177983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8230e-c0cc-486e-9781-966d8633b986 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:31.190353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 185/373] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:31.192691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4143dd-3975-4383-be04-c4bbc9582ae2 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.193478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4143dd-3975-4383-be04-c4bbc9582ae2 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.193970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4143dd-3975-4383-be04-c4bbc9582ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.199615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58d9dc-dd61-4161-ad20-7336201a187e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.217439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0c4143dd-3975-4383-be04-c4bbc9582ae2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:31.218535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0c4143dd-3975-4383-be04-c4bbc9582ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:31.241779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed9d51-ca9c-4488-adb9-8c93ccf9102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.242679 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed9d51-ca9c-4488-adb9-8c93ccf9102e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.243916 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed9d51-ca9c-4488-adb9-8c93ccf9102e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.243916 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed9d51-ca9c-4488-adb9-8c93ccf9102e None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.247452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed9d51-ca9c-4488-adb9-8c93ccf9102e None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:31.247452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed9d51-ca9c-4488-adb9-8c93ccf9102e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:31.248954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 189/374] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:31.249533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed9d51-ca9c-4488-adb9-8c93ccf9102e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.270654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:31.314994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 186/375] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:31.314994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4143dd-3975-4383-be04-c4bbc9582ae2 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.346635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:31.391828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed9d51-ca9c-4488-adb9-8c93ccf9102e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.442219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.443097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.443843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.444324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.445452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] RBAC: Adding query filter params (group.id=8c8057e0f50d4775902e859951a8277c, role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=f618dc420f704f2c8a85cfe182a07300) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:31.446324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:31.449145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:31.451053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4143dd-3975-4383-be04-c4bbc9582ae2 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.457121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 190/376] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/role_assignments?group.id=8c8057e0f50d4775902e859951a8277c&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:31.458104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.531786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003c94d-8b4b-424d-ad1d-df8d54e79457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.532596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003c94d-8b4b-424d-ad1d-df8d54e79457 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.532918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003c94d-8b4b-424d-ad1d-df8d54e79457 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.533189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003c94d-8b4b-424d-ad1d-df8d54e79457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.605425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:31.637298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003c94d-8b4b-424d-ad1d-df8d54e79457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:31.638780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003c94d-8b4b-424d-ad1d-df8d54e79457 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:31.677467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f0d53-2b29-492e-961e-99ea6659dc03 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.715556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db7a3-368a-475d-8015-298c335b9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.716201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db7a3-368a-475d-8015-298c335b9d3d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.716518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db7a3-368a-475d-8015-298c335b9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.716786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db7a3-368a-475d-8015-298c335b9d3d None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.717922 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336db7a3-368a-475d-8015-298c335b9d3d None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:31.720104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336db7a3-368a-475d-8015-298c335b9d3d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:31.722242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 191/377] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:31.722775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db7a3-368a-475d-8015-298c335b9d3d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.730259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:31.771585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 187/378] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:31.772858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003c94d-8b4b-424d-ad1d-df8d54e79457 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.865470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db7a3-368a-475d-8015-298c335b9d3d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.870000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:31.906694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003c94d-8b4b-424d-ad1d-df8d54e79457 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:31.913616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91a005-7734-469b-bba5-5a9cd12956c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.914559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91a005-7734-469b-bba5-5a9cd12956c1 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.915161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91a005-7734-469b-bba5-5a9cd12956c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.917276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91a005-7734-469b-bba5-5a9cd12956c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.930300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91a005-7734-469b-bba5-5a9cd12956c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-4d91a005-7734-469b-bba5-5a9cd12956c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:31.937979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:31.938988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:31.939786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:31.945450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 192/379] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:31.946566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91a005-7734-469b-bba5-5a9cd12956c1 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:31.947524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:31.948339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:31.948677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:31.949050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:31.949795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] RBAC: Adding query filter params (group.id=8c8057e0f50d4775902e859951a8277c, role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=f618dc420f704f2c8a85cfe182a07300) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:31.950159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:31.952890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:31.959106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 188/380] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/role_assignments?group.id=8c8057e0f50d4775902e859951a8277c&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:31.960583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:32.038879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:32.099205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b4b08-6fb9-4626-9148-74d5a7a09ccc None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.103143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91a005-7734-469b-bba5-5a9cd12956c1 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.142488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.143308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.144011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.144283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.145016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:32.145318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.146735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.162680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 189/381] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:32.168575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:32.171527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:32.175477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc858d-0a7e-4669-9385-408e43ece31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.176399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc858d-0a7e-4669-9385-408e43ece31a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.176871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc858d-0a7e-4669-9385-408e43ece31a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.177337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc858d-0a7e-4669-9385-408e43ece31a None admin] PATH_INFO: `/v3/projects/27572d94c00243fdbbfd37ae021fbe77` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.178683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc858d-0a7e-4669-9385-408e43ece31a None admin] RBAC: Authorizing `identity:get_project(project_id=27572d94c00243fdbbfd37ae021fbe77)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.180754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc858d-0a7e-4669-9385-408e43ece31a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.183575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 193/382] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:31 2022] GET /identity/v3/projects/27572d94c00243fdbbfd37ae021fbe77 => generated 344 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:32.191966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc858d-0a7e-4669-9385-408e43ece31a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373998e-600e-4051-9276-0e3ad96db270 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.347744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc858d-0a7e-4669-9385-408e43ece31a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.393315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:32.399553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b5abb-be8b-424f-9cce-3ad3e7dc2889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.400218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5abb-be8b-424f-9cce-3ad3e7dc2889 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.400496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5abb-be8b-424f-9cce-3ad3e7dc2889 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.400870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5abb-be8b-424f-9cce-3ad3e7dc2889 None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.406577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2121f8-c609-4d7c-ad2e-0082f21620a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.406577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2121f8-c609-4d7c-ad2e-0082f21620a5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.406577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2121f8-c609-4d7c-ad2e-0082f21620a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.406577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2121f8-c609-4d7c-ad2e-0082f21620a5 None admin] PATH_INFO: `/v3/groups/09804aa6302d4ed182f5b3eb519232ad` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.406867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b5abb-be8b-424f-9cce-3ad3e7dc2889 None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.406867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b5abb-be8b-424f-9cce-3ad3e7dc2889 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.407814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 190/383] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:32 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:32.408796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24550708-02d7-450c-90ec-863b8d26a610 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.409076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24550708-02d7-450c-90ec-863b8d26a610 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.409343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24550708-02d7-450c-90ec-863b8d26a610 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.410545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 191/384] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:32.418399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2121f8-c609-4d7c-ad2e-0082f21620a5 None admin] RBAC: Authorizing `identity:get_group(group_id=09804aa6302d4ed182f5b3eb519232ad)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.419825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfff886-b73f-4572-bdf2-70f5a122d052 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.419936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2121f8-c609-4d7c-ad2e-0082f21620a5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.420432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfff886-b73f-4572-bdf2-70f5a122d052 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.420904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfff886-b73f-4572-bdf2-70f5a122d052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.421826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 194/385] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:32 2022] GET /identity/v3/groups/09804aa6302d4ed182f5b3eb519232ad => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:32.428508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2121f8-c609-4d7c-ad2e-0082f21620a5 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:32.448494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8bfff886-b73f-4572-bdf2-70f5a122d052 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:32.450224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8bfff886-b73f-4572-bdf2-70f5a122d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:32.543882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:32.551389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 192/386] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:32.552348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfff886-b73f-4572-bdf2-70f5a122d052 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:32.595414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2121f8-c609-4d7c-ad2e-0082f21620a5 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.649311 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ae5c5-8d92-49a2-8cad-34925e256358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.650030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ae5c5-8d92-49a2-8cad-34925e256358 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.650304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ae5c5-8d92-49a2-8cad-34925e256358 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.650570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ae5c5-8d92-49a2-8cad-34925e256358 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.653224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ae5c5-8d92-49a2-8cad-34925e256358 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.654671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ae5c5-8d92-49a2-8cad-34925e256358 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.657365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 195/387] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:32 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:32.663518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ae5c5-8d92-49a2-8cad-34925e256358 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:32.737055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfff886-b73f-4572-bdf2-70f5a122d052 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.739249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:32.784993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ae5c5-8d92-49a2-8cad-34925e256358 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:32.826055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30886c14-d55a-41f7-a29e-dfd066323eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.826736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886c14-d55a-41f7-a29e-dfd066323eeb None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.827053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886c14-d55a-41f7-a29e-dfd066323eeb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.827502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886c14-d55a-41f7-a29e-dfd066323eeb None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/groups/8c8057e0f50d4775902e859951a8277c/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.829494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30886c14-d55a-41f7-a29e-dfd066323eeb None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c, group_id=8c8057e0f50d4775902e859951a8277c, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.830941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30886c14-d55a-41f7-a29e-dfd066323eeb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.836643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:32.837346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:32.837621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:32.837888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:32.839367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] RBAC: Adding query filter params (group.id=09804aa6302d4ed182f5b3eb519232ad, role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=27572d94c00243fdbbfd37ae021fbe77) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:32.839666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:32.841414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:32.848238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 193/388] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:32 2022] GET /identity/v3/role_assignments?group.id=09804aa6302d4ed182f5b3eb519232ad&scope.project.id=27572d94c00243fdbbfd37ae021fbe77&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 260 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:32.982656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 196/389] 10.222.0.94 () {58 vars in 1442 bytes} [Fri Aug 26 15:35:32 2022] PUT /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/groups/8c8057e0f50d4775902e859951a8277c/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:35:33.029595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:33.129657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:33.187987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741479e2-709a-4f4d-8f45-e40435dc8e74 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:33.229163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ceb761-f303-47da-89bd-1ed4ab6336b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:33.230076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ceb761-f303-47da-89bd-1ed4ab6336b1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:33.230356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ceb761-f303-47da-89bd-1ed4ab6336b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:33.230627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ceb761-f303-47da-89bd-1ed4ab6336b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:33.243571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ceb761-f303-47da-89bd-1ed4ab6336b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-37ceb761-f303-47da-89bd-1ed4ab6336b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:33.249630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:33.250569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:33.251407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 194/390] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:33.258669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30886c14-d55a-41f7-a29e-dfd066323eeb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:33.376465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30886c14-d55a-41f7-a29e-dfd066323eeb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:33.407302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:33.414444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:33.420326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:33.420326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:33.420326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:33.420326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:33.420326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:33.420326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:33.428097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 197/391] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:33.433911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ceb761-f303-47da-89bd-1ed4ab6336b1 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:33.568242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ceb761-f303-47da-89bd-1ed4ab6336b1 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:33.595462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:33.614713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f013585-42d2-459d-9c37-7b6a6291cc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:33.615404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f013585-42d2-459d-9c37-7b6a6291cc27 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:33.615690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f013585-42d2-459d-9c37-7b6a6291cc27 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:33.615959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f013585-42d2-459d-9c37-7b6a6291cc27 None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:33.617448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f013585-42d2-459d-9c37-7b6a6291cc27 None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:33.618903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f013585-42d2-459d-9c37-7b6a6291cc27 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:33.624430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 195/392] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:33 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:33.630062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:33.753495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1040886-b351-4d7b-b12c-7f653a87d8d4 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:33.753839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1040886-b351-4d7b-b12c-7f653a87d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:33.754044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1040886-b351-4d7b-b12c-7f653a87d8d4 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:33.755706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 196/393] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:33.756883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec122b0a-4a8f-4966-906e-40e9911991b7 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:33.762347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160495a8-8e34-4cf3-a6b2-33a94d4ed677 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:33.762637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160495a8-8e34-4cf3-a6b2-33a94d4ed677 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:33.762936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160495a8-8e34-4cf3-a6b2-33a94d4ed677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:33.786330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-160495a8-8e34-4cf3-a6b2-33a94d4ed677 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:33.787749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-160495a8-8e34-4cf3-a6b2-33a94d4ed677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:33.820049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c378fe-e577-4ad7-a709-c686a5785c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:33.820690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c378fe-e577-4ad7-a709-c686a5785c56 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:33.820967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c378fe-e577-4ad7-a709-c686a5785c56 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:33.821235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c378fe-e577-4ad7-a709-c686a5785c56 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:33.823878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c378fe-e577-4ad7-a709-c686a5785c56 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:33.826880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c378fe-e577-4ad7-a709-c686a5785c56 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:33.828778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 198/394] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:33 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:33.834126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c378fe-e577-4ad7-a709-c686a5785c56 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:33.911451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:33.961289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 197/395] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:33.961899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c378fe-e577-4ad7-a709-c686a5785c56 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:34.000993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce6306-e6eb-47ce-84d6-3adaf8101f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:34.001659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6306-e6eb-47ce-84d6-3adaf8101f3f None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:34.001930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6306-e6eb-47ce-84d6-3adaf8101f3f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:34.002198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce6306-e6eb-47ce-84d6-3adaf8101f3f None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/groups/8c8057e0f50d4775902e859951a8277c/roles/98aaf5af7bb24b1596a9b01fd76eaf2c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:34.004453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce6306-e6eb-47ce-84d6-3adaf8101f3f None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c, group_id=8c8057e0f50d4775902e859951a8277c, role_id=98aaf5af7bb24b1596a9b01fd76eaf2c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:34.005909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce6306-e6eb-47ce-84d6-3adaf8101f3f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:34.101479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 199/396] 10.222.0.94 () {58 vars in 1442 bytes} [Fri Aug 26 15:35:33 2022] PUT /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c/groups/8c8057e0f50d4775902e859951a8277c/roles/98aaf5af7bb24b1596a9b01fd76eaf2c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:34.265841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160495a8-8e34-4cf3-a6b2-33a94d4ed677 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:34.394769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f1de1-ad64-4f47-9d06-9d445f152217 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:34.395100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f1de1-ad64-4f47-9d06-9d445f152217 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:34.395373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f1de1-ad64-4f47-9d06-9d445f152217 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:34.397013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 200/397] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:34.403737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a5ee1-738f-40f1-9ecb-bd40391fcaaf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:34.404012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a5ee1-738f-40f1-9ecb-bd40391fcaaf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:34.404278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a5ee1-738f-40f1-9ecb-bd40391fcaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:34.408928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160495a8-8e34-4cf3-a6b2-33a94d4ed677 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:34.425914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e78a5ee1-738f-40f1-9ecb-bd40391fcaaf None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:34.427557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e78a5ee1-738f-40f1-9ecb-bd40391fcaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:34.443458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:34.452483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb64fc5-2347-43fe-965d-9ceb19220fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:34.453475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb64fc5-2347-43fe-965d-9ceb19220fd7 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:34.456417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb64fc5-2347-43fe-965d-9ceb19220fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:34.456417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb64fc5-2347-43fe-965d-9ceb19220fd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:34.486226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb64fc5-2347-43fe-965d-9ceb19220fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ecb64fc5-2347-43fe-965d-9ceb19220fd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:34.491142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:34.492281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:34.493130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 198/398] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:34.499215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb64fc5-2347-43fe-965d-9ceb19220fd7 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:34.529130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 201/399] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 26 15:35:34.633943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb64fc5-2347-43fe-965d-9ceb19220fd7 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:34.687404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:34.688021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:34.688300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:34.688570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:34.689243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:34.689544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:34.690915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:34.700499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 199/400] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:34.706198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a5ee1-738f-40f1-9ecb-bd40391fcaaf None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:34.777601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:34.856100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:34.866190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a5ee1-738f-40f1-9ecb-bd40391fcaaf None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:34.907989 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa6590-e99e-4eb2-a12f-87706edfcaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:34.909007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa6590-e99e-4eb2-a12f-87706edfcaa9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:34.909484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa6590-e99e-4eb2-a12f-87706edfcaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:34.909960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa6590-e99e-4eb2-a12f-87706edfcaa9 None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:34.911451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa6590-e99e-4eb2-a12f-87706edfcaa9 None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:34.913268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa6590-e99e-4eb2-a12f-87706edfcaa9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:34.915410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 202/401] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:34 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:34.928910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:34.930899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa6590-e99e-4eb2-a12f-87706edfcaa9 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:34.989419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34463c03-a7e1-4444-9a54-707ed2a8977d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:35.048970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa6590-e99e-4eb2-a12f-87706edfcaa9 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:35.053670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a584d-97cb-46cd-9645-598eccbefccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:35.054322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a584d-97cb-46cd-9645-598eccbefccb None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.054600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a584d-97cb-46cd-9645-598eccbefccb None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.054872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a584d-97cb-46cd-9645-598eccbefccb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.066317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:35.076885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a584d-97cb-46cd-9645-598eccbefccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-9e2a584d-97cb-46cd-9645-598eccbefccb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:35.082225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:35.083163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:35.083932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:35.091850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 200/402] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:35.095335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf28695-bb4c-4750-be1b-062d2d800d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:35.096507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf28695-bb4c-4750-be1b-062d2d800d9a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.097045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf28695-bb4c-4750-be1b-062d2d800d9a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.097149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a584d-97cb-46cd-9645-598eccbefccb None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:35.097735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf28695-bb4c-4750-be1b-062d2d800d9a None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.099215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf28695-bb4c-4750-be1b-062d2d800d9a None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:35.101856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf28695-bb4c-4750-be1b-062d2d800d9a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:35.104909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 203/403] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:34 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:35.117647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf28695-bb4c-4750-be1b-062d2d800d9a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:35.224251 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a584d-97cb-46cd-9645-598eccbefccb None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:35.226630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:35.236724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf28695-bb4c-4750-be1b-062d2d800d9a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:35.267436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:35.268027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.268309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.268576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.269252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:35.269702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:35.270956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:35.279188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a00fd-5a23-42e2-baaf-0d4510763f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:35.280182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a00fd-5a23-42e2-baaf-0d4510763f8b None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.280667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a00fd-5a23-42e2-baaf-0d4510763f8b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.281185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a00fd-5a23-42e2-baaf-0d4510763f8b None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300/groups/8c8057e0f50d4775902e859951a8277c/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.283525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a00fd-5a23-42e2-baaf-0d4510763f8b None admin] RBAC: Authorizing `identity:create_grant(project_id=f618dc420f704f2c8a85cfe182a07300, group_id=8c8057e0f50d4775902e859951a8277c, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:35.285234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a00fd-5a23-42e2-baaf-0d4510763f8b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:35.289769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 201/404] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:35.295677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:35.381650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 204/405] 10.222.0.94 () {58 vars in 1442 bytes} [Fri Aug 26 15:35:35 2022] PUT /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300/groups/8c8057e0f50d4775902e859951a8277c/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:35:35.415597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0a97-9344-46ad-9bdc-6346178db6cc None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.415954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0a97-9344-46ad-9bdc-6346178db6cc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.416301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0a97-9344-46ad-9bdc-6346178db6cc None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.418744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 205/406] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:35.419755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be1508-7767-49af-98ef-43b052f39cd8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:35.424361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e161c-aa77-4495-a6ed-2de058839e64 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.428664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e161c-aa77-4495-a6ed-2de058839e64 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.428664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e161c-aa77-4495-a6ed-2de058839e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.446947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-f46e161c-aa77-4495-a6ed-2de058839e64 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:35.448840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f46e161c-aa77-4495-a6ed-2de058839e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:35.489417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eaf7d-ae69-4d5d-b381-329c74ff9bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:35.490127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eaf7d-ae69-4d5d-b381-329c74ff9bb7 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.490410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eaf7d-ae69-4d5d-b381-329c74ff9bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.490685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eaf7d-ae69-4d5d-b381-329c74ff9bb7 None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.493221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726eaf7d-ae69-4d5d-b381-329c74ff9bb7 None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:35.497691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726eaf7d-ae69-4d5d-b381-329c74ff9bb7 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:35.499514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 202/407] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:35 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:35.512073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eaf7d-ae69-4d5d-b381-329c74ff9bb7 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:35.531139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:35.566170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:35.579413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 206/408] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:35.649637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eaf7d-ae69-4d5d-b381-329c74ff9bb7 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:35.696156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41401ec-eec4-477b-8d9c-25dfacba9ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:35.697167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41401ec-eec4-477b-8d9c-25dfacba9ce4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.698546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41401ec-eec4-477b-8d9c-25dfacba9ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.699152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41401ec-eec4-477b-8d9c-25dfacba9ce4 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.703734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41401ec-eec4-477b-8d9c-25dfacba9ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:35.705560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41401ec-eec4-477b-8d9c-25dfacba9ce4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:35.707846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 203/409] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:35 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:35.714349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e161c-aa77-4495-a6ed-2de058839e64 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:35.805274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062d099-5b6c-4eee-a9a2-fe40a3554b05 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.805922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062d099-5b6c-4eee-a9a2-fe40a3554b05 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.806412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062d099-5b6c-4eee-a9a2-fe40a3554b05 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.809032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 204/410] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:35.813800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb544d-2181-48cb-9bd3-9e514cb6f9fe None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.814326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb544d-2181-48cb-9bd3-9e514cb6f9fe None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.814871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb544d-2181-48cb-9bd3-9e514cb6f9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.851254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-33fb544d-2181-48cb-9bd3-9e514cb6f9fe None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:35.851462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e161c-aa77-4495-a6ed-2de058839e64 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:35.852920 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-33fb544d-2181-48cb-9bd3-9e514cb6f9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:35.904880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8cfcc-f5a7-4487-a421-6e4ca0dac4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:35.906271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8cfcc-f5a7-4487-a421-6e4ca0dac4d4 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:35.907339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8cfcc-f5a7-4487-a421-6e4ca0dac4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:35.908918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8cfcc-f5a7-4487-a421-6e4ca0dac4d4 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300/groups/8c8057e0f50d4775902e859951a8277c/roles/98aaf5af7bb24b1596a9b01fd76eaf2c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:35.914501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8cfcc-f5a7-4487-a421-6e4ca0dac4d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f618dc420f704f2c8a85cfe182a07300, group_id=8c8057e0f50d4775902e859951a8277c, role_id=98aaf5af7bb24b1596a9b01fd76eaf2c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:35.920101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8cfcc-f5a7-4487-a421-6e4ca0dac4d4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:35.956087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 205/411] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:35.957468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb544d-2181-48cb-9bd3-9e514cb6f9fe None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:35.982005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 207/412] 10.222.0.94 () {58 vars in 1442 bytes} [Fri Aug 26 15:35:35 2022] PUT /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300/groups/8c8057e0f50d4775902e859951a8277c/roles/98aaf5af7bb24b1596a9b01fd76eaf2c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:36.010462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:36.109801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb544d-2181-48cb-9bd3-9e514cb6f9fe None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:36.165131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480977f-7c70-4a77-9ae4-a691195c0873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:36.165709 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480977f-7c70-4a77-9ae4-a691195c0873 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:36.165987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480977f-7c70-4a77-9ae4-a691195c0873 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:36.166446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480977f-7c70-4a77-9ae4-a691195c0873 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:36.184177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480977f-7c70-4a77-9ae4-a691195c0873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-7480977f-7c70-4a77-9ae4-a691195c0873 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:36.189063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:36.189969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:36.190782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 206/413] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:36.197548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8cfcc-f5a7-4487-a421-6e4ca0dac4d4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:36.270259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:36.299219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480977f-7c70-4a77-9ae4-a691195c0873 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:36.369771 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8cfcc-f5a7-4487-a421-6e4ca0dac4d4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:36.410941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:36.411697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:36.414160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:36.414160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:36.414160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:36.414160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:36.414656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:36.420530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:36.425251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 208/414] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:36.430553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480977f-7c70-4a77-9ae4-a691195c0873 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:36.430742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:36.471492 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948ec31-ff26-4e67-a5c2-ba149d232e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:36.472142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948ec31-ff26-4e67-a5c2-ba149d232e53 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:36.472420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948ec31-ff26-4e67-a5c2-ba149d232e53 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:36.472695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948ec31-ff26-4e67-a5c2-ba149d232e53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:36.485216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948ec31-ff26-4e67-a5c2-ba149d232e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-8948ec31-ff26-4e67-a5c2-ba149d232e53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:36.490426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:36.491360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:36.492404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 207/415] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:36.498493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948ec31-ff26-4e67-a5c2-ba149d232e53 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:36.626270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948ec31-ff26-4e67-a5c2-ba149d232e53 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:36.651461 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:36.661967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94f76c-3d7f-45dc-b948-fe83cf379734 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:36.671586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30384858-d670-4550-906e-5c9b887c63f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:36.672212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30384858-d670-4550-906e-5c9b887c63f6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:36.672406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30384858-d670-4550-906e-5c9b887c63f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:36.673058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30384858-d670-4550-906e-5c9b887c63f6 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:36.673738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30384858-d670-4550-906e-5c9b887c63f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:36.674341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30384858-d670-4550-906e-5c9b887c63f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:36.676093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30384858-d670-4550-906e-5c9b887c63f6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:36.688714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 208/416] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:36.694856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48902f19-8d7d-42b1-95c3-92e2d2071b53 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:36.695338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48902f19-8d7d-42b1-95c3-92e2d2071b53 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:36.695623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48902f19-8d7d-42b1-95c3-92e2d2071b53 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:36.697181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 209/417] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:36.699419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48902f19-8d7d-42b1-95c3-92e2d2071b53 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:36.708470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432be0ca-1caa-4c39-ab29-452b1d13c7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:36.709170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432be0ca-1caa-4c39-ab29-452b1d13c7a0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:36.709443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432be0ca-1caa-4c39-ab29-452b1d13c7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:36.709709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432be0ca-1caa-4c39-ab29-452b1d13c7a0 None admin] PATH_INFO: `/v3/groups/09804aa6302d4ed182f5b3eb519232ad` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:36.711059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432be0ca-1caa-4c39-ab29-452b1d13c7a0 None admin] RBAC: Authorizing `identity:get_group(group_id=09804aa6302d4ed182f5b3eb519232ad)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:36.712647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432be0ca-1caa-4c39-ab29-452b1d13c7a0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:36.715004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 209/418] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:36 2022] GET /identity/v3/groups/09804aa6302d4ed182f5b3eb519232ad => generated 229 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:36.716287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0af4-ff3c-4b40-95fc-32b10494c286 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:36.716563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0af4-ff3c-4b40-95fc-32b10494c286 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:36.716828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0af4-ff3c-4b40-95fc-32b10494c286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:36.719425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:36.740033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-96fe0af4-ff3c-4b40-95fc-32b10494c286 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:36.742012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-96fe0af4-ff3c-4b40-95fc-32b10494c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:36.827455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48902f19-8d7d-42b1-95c3-92e2d2071b53 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:36.839039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 210/419] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:36.840237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0af4-ff3c-4b40-95fc-32b10494c286 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:36.911657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8a40d-83b3-47aa-b269-ce12cd70be7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:36.912291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8a40d-83b3-47aa-b269-ce12cd70be7b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:36.912573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8a40d-83b3-47aa-b269-ce12cd70be7b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:36.912843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8a40d-83b3-47aa-b269-ce12cd70be7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:36.972823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:36.994875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0af4-ff3c-4b40-95fc-32b10494c286 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:37.032927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8a40d-83b3-47aa-b269-ce12cd70be7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:37.035557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8a40d-83b3-47aa-b269-ce12cd70be7b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:37.066591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:37.068212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfa7e9-5c8a-4dfa-ad9c-dc8eb4114531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:37.068854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfa7e9-5c8a-4dfa-ad9c-dc8eb4114531 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.069128 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfa7e9-5c8a-4dfa-ad9c-dc8eb4114531 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.069395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfa7e9-5c8a-4dfa-ad9c-dc8eb4114531 None admin] PATH_INFO: `/v3/projects/27572d94c00243fdbbfd37ae021fbe77` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.070781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfa7e9-5c8a-4dfa-ad9c-dc8eb4114531 None admin] RBAC: Authorizing `identity:get_project(project_id=27572d94c00243fdbbfd37ae021fbe77)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:37.072252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfa7e9-5c8a-4dfa-ad9c-dc8eb4114531 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:37.074374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 211/420] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:36 2022] GET /identity/v3/projects/27572d94c00243fdbbfd37ae021fbe77 => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:37.080834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfa7e9-5c8a-4dfa-ad9c-dc8eb4114531 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:37.191182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfa7e9-5c8a-4dfa-ad9c-dc8eb4114531 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:37.228408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79bf30-2396-4c7a-8f01-79b0585ed525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:37.230003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79bf30-2396-4c7a-8f01-79b0585ed525 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.230464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79bf30-2396-4c7a-8f01-79b0585ed525 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.230803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79bf30-2396-4c7a-8f01-79b0585ed525 None admin] PATH_INFO: `/v3/projects/27572d94c00243fdbbfd37ae021fbe77/groups/09804aa6302d4ed182f5b3eb519232ad/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.230899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 210/421] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:37.231848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8a40d-83b3-47aa-b269-ce12cd70be7b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:37.232961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79bf30-2396-4c7a-8f01-79b0585ed525 None admin] RBAC: Authorizing `identity:create_grant(project_id=27572d94c00243fdbbfd37ae021fbe77, group_id=09804aa6302d4ed182f5b3eb519232ad, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:37.234408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79bf30-2396-4c7a-8f01-79b0585ed525 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:37.309339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 212/422] 10.222.0.94 () {58 vars in 1442 bytes} [Fri Aug 26 15:35:37 2022] PUT /identity/v3/projects/27572d94c00243fdbbfd37ae021fbe77/groups/09804aa6302d4ed182f5b3eb519232ad/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:35:37.316060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79bf30-2396-4c7a-8f01-79b0585ed525 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:37.346291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8a40d-83b3-47aa-b269-ce12cd70be7b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:37.354229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:37.364620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:37.390095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dead2f-3354-40e6-bcd9-1f28beb7b36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:37.390860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dead2f-3354-40e6-bcd9-1f28beb7b36d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.391313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dead2f-3354-40e6-bcd9-1f28beb7b36d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.391692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dead2f-3354-40e6-bcd9-1f28beb7b36d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.419635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dead2f-3354-40e6-bcd9-1f28beb7b36d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-b1dead2f-3354-40e6-bcd9-1f28beb7b36d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:37.427547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:37.428495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:37.429266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:37.432514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 211/423] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:37.441681 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dead2f-3354-40e6-bcd9-1f28beb7b36d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:37.530092 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79bf30-2396-4c7a-8f01-79b0585ed525 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:37.591284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbb4f3-3789-4da3-87d3-941ce8da0bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:37.592273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbb4f3-3789-4da3-87d3-941ce8da0bfb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.593979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbb4f3-3789-4da3-87d3-941ce8da0bfb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.594471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbb4f3-3789-4da3-87d3-941ce8da0bfb None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.595957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbb4f3-3789-4da3-87d3-941ce8da0bfb None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:37.597662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbb4f3-3789-4da3-87d3-941ce8da0bfb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:37.599573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 213/424] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:37 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:37.605017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbb4f3-3789-4da3-87d3-941ce8da0bfb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:37.654363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dead2f-3354-40e6-bcd9-1f28beb7b36d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:37.724041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:37.724686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.724949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.725184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.725867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:37.726164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:37.727587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:37.739373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:37.747379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 212/425] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:37.752828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:37.759554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbb4f3-3789-4da3-87d3-941ce8da0bfb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:37.805073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:37.807477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbb407-b5f3-4d4a-8c93-9bcfb6c8b770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:37.808315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbb407-b5f3-4d4a-8c93-9bcfb6c8b770 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.808739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbb407-b5f3-4d4a-8c93-9bcfb6c8b770 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.809174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbb407-b5f3-4d4a-8c93-9bcfb6c8b770 None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.810574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbb407-b5f3-4d4a-8c93-9bcfb6c8b770 None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:37.812773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbb407-b5f3-4d4a-8c93-9bcfb6c8b770 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:37.815900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 214/426] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:37 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:37.820903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a693b40-0f62-4e37-8d76-092b0d0b72cc None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.820903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a693b40-0f62-4e37-8d76-092b0d0b72cc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.820903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a693b40-0f62-4e37-8d76-092b0d0b72cc None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.822228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 215/427] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:37.825056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a693b40-0f62-4e37-8d76-092b0d0b72cc None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:37.893437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec138b-4bff-4e1f-8d9e-fc850b5e42a4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:37.936560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbbbc1-6104-4b17-8ad7-baf1dcbbff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:37.938085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbbbc1-6104-4b17-8ad7-baf1dcbbff72 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.938703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbbbc1-6104-4b17-8ad7-baf1dcbbff72 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.939205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbbbc1-6104-4b17-8ad7-baf1dcbbff72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.941265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a693b40-0f62-4e37-8d76-092b0d0b72cc None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:37.981719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe2ade-fc69-478c-8236-ca5078f8fbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:37.982376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe2ade-fc69-478c-8236-ca5078f8fbbd None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.982650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe2ade-fc69-478c-8236-ca5078f8fbbd None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.982911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe2ade-fc69-478c-8236-ca5078f8fbbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:37.983691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe2ade-fc69-478c-8236-ca5078f8fbbd None admin] RBAC: Adding query filter params (group.id=8c8057e0f50d4775902e859951a8277c, role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:37.984120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe2ade-fc69-478c-8236-ca5078f8fbbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:37.985694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe2ade-fc69-478c-8236-ca5078f8fbbd None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:37.995015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 216/428] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:37 2022] GET /identity/v3/role_assignments?group.id=8c8057e0f50d4775902e859951a8277c&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:37.995015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d649ca-feb6-4892-b876-64cd628eb6b1 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:37.995015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d649ca-feb6-4892-b876-64cd628eb6b1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:37.995015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d649ca-feb6-4892-b876-64cd628eb6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:38.027948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-23d649ca-feb6-4892-b876-64cd628eb6b1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:38.029155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-23d649ca-feb6-4892-b876-64cd628eb6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:38.044913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbbbc1-6104-4b17-8ad7-baf1dcbbff72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:38.047219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbbbc1-6104-4b17-8ad7-baf1dcbbff72 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:38.068302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:38.108753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:38.161639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 217/429] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:38.222166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 213/430] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 15:35:38.227879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d649ca-feb6-4892-b876-64cd628eb6b1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:38.362238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d649ca-feb6-4892-b876-64cd628eb6b1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:38.402365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d3eab-c7f2-4f86-a9e0-ccbb0cf9d519 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:38.402764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d3eab-c7f2-4f86-a9e0-ccbb0cf9d519 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:38.402988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d3eab-c7f2-4f86-a9e0-ccbb0cf9d519 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:38.404533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 214/431] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:38.410663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04590da1-db93-41a2-93fe-2ac9fc9e73f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:38.412901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a86f63-5e3a-4a3b-97f3-b2bf4ae9997c None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:38.413003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04590da1-db93-41a2-93fe-2ac9fc9e73f2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:38.413076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a86f63-5e3a-4a3b-97f3-b2bf4ae9997c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:38.413161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04590da1-db93-41a2-93fe-2ac9fc9e73f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:38.413238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a86f63-5e3a-4a3b-97f3-b2bf4ae9997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:38.413314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04590da1-db93-41a2-93fe-2ac9fc9e73f2 None admin] PATH_INFO: `/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:38.414136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04590da1-db93-41a2-93fe-2ac9fc9e73f2 None admin] RBAC: Authorizing `identity:get_project(project_id=6a4f8e5f80a549d7b0f88b8e177f9e5c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:38.415945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04590da1-db93-41a2-93fe-2ac9fc9e73f2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:38.419561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 218/432] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:38 2022] GET /identity/v3/projects/6a4f8e5f80a549d7b0f88b8e177f9e5c => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:38.428977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04590da1-db93-41a2-93fe-2ac9fc9e73f2 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:38.442201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-15a86f63-5e3a-4a3b-97f3-b2bf4ae9997c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:38.443435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-15a86f63-5e3a-4a3b-97f3-b2bf4ae9997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:38.450404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:38.533070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 215/433] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 26 15:35:38.533740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a86f63-5e3a-4a3b-97f3-b2bf4ae9997c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:38.563585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04590da1-db93-41a2-93fe-2ac9fc9e73f2 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:38.618795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8faf2e-db78-4c32-b04b-d89fbdf0948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:38.620706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8faf2e-db78-4c32-b04b-d89fbdf0948f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:38.620982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8faf2e-db78-4c32-b04b-d89fbdf0948f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:38.621253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8faf2e-db78-4c32-b04b-d89fbdf0948f None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:38.622423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8faf2e-db78-4c32-b04b-d89fbdf0948f None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:38.623899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8faf2e-db78-4c32-b04b-d89fbdf0948f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:38.624640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:38.627781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 219/434] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:38 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:38.633519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8faf2e-db78-4c32-b04b-d89fbdf0948f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:38.671652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a86f63-5e3a-4a3b-97f3-b2bf4ae9997c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:38.709898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b461f-5b22-4371-92d4-defac9aa3b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:38.710890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b461f-5b22-4371-92d4-defac9aa3b83 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:38.711435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b461f-5b22-4371-92d4-defac9aa3b83 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:38.711917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b461f-5b22-4371-92d4-defac9aa3b83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:38.723240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b461f-5b22-4371-92d4-defac9aa3b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-6a5b461f-5b22-4371-92d4-defac9aa3b83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:38.728261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:38.730206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 15:35:38.731017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:38.733693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 216/435] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:38.737084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:38.739292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b461f-5b22-4371-92d4-defac9aa3b83 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:38.756319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8faf2e-db78-4c32-b04b-d89fbdf0948f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:38.797891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:38.798512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:38.798791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:38.799146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:38.799991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] RBAC: Adding query filter params (group.id=8c8057e0f50d4775902e859951a8277c, role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:38.800350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:38.801841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:38.815505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 220/436] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:38 2022] GET /identity/v3/role_assignments?group.id=8c8057e0f50d4775902e859951a8277c&scope.project.id=6a4f8e5f80a549d7b0f88b8e177f9e5c&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:38.867598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b461f-5b22-4371-92d4-defac9aa3b83 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:38.868320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:38.908363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:38.915893 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:38.918022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:38.918535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:38.919014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:38.920316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:38.925398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:38.925398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:38.935828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 217/437] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:35:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:38.942125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:38.998915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97ca67-2b5b-4311-a352-34f62c11b291 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.045920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6e5ca-1e71-471e-9c68-5fc2c0458890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:39.047403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6e5ca-1e71-471e-9c68-5fc2c0458890 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.047883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6e5ca-1e71-471e-9c68-5fc2c0458890 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.048368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6e5ca-1e71-471e-9c68-5fc2c0458890 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.066783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6e5ca-1e71-471e-9c68-5fc2c0458890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-09c6e5ca-1e71-471e-9c68-5fc2c0458890 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:39.073665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:39.075413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:39.077678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 221/438] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:39.081199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74540e97-dbcd-45c1-87e5-df806b2d2ee6 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.081645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6e5ca-1e71-471e-9c68-5fc2c0458890 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:39.098574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:39.128280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535548a-58a2-4391-b888-4801754a1ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:39.129080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535548a-58a2-4391-b888-4801754a1ee7 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.129420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535548a-58a2-4391-b888-4801754a1ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.129724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535548a-58a2-4391-b888-4801754a1ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.224358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6e5ca-1e71-471e-9c68-5fc2c0458890 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.226443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:39.230212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535548a-58a2-4391-b888-4801754a1ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:39.231891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535548a-58a2-4391-b888-4801754a1ee7 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:39.266002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:39.266616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.266886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.267190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.267852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:39.268161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:39.269516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:39.281428 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 222/439] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:39.286875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:39.367677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 218/440] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:39.373921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535548a-58a2-4391-b888-4801754a1ee7 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:39.384113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:39.433283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e53c9-8274-4465-843c-69eadc77ed5d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.469997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653978a-7434-401b-9ea1-9818e7ced084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:39.470610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653978a-7434-401b-9ea1-9818e7ced084 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.470881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653978a-7434-401b-9ea1-9818e7ced084 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.471193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653978a-7434-401b-9ea1-9818e7ced084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.484067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:39.498237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535548a-58a2-4391-b888-4801754a1ee7 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.535327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d15567-fad9-4caf-93d3-17bdaf758075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:39.535953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d15567-fad9-4caf-93d3-17bdaf758075 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.536225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d15567-fad9-4caf-93d3-17bdaf758075 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.536491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d15567-fad9-4caf-93d3-17bdaf758075 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.537607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d15567-fad9-4caf-93d3-17bdaf758075 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:39.539065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d15567-fad9-4caf-93d3-17bdaf758075 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:39.540851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 219/441] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:39 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:39.546029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d15567-fad9-4caf-93d3-17bdaf758075 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:39.547771 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653978a-7434-401b-9ea1-9818e7ced084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:39.549252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653978a-7434-401b-9ea1-9818e7ced084 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:39.630564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:39.656784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d15567-fad9-4caf-93d3-17bdaf758075 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.662878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 223/442] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:39.664454 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00094fd-f3f6-402a-97ed-7d77c17f4cc3 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.665185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00094fd-f3f6-402a-97ed-7d77c17f4cc3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.665783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00094fd-f3f6-402a-97ed-7d77c17f4cc3 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.670741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 224/443] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:39.671570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00094fd-f3f6-402a-97ed-7d77c17f4cc3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:39.692583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032b3d0-1e5d-42f5-9c46-47cd7dd8060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:39.693219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b3d0-1e5d-42f5-9c46-47cd7dd8060c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.693498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b3d0-1e5d-42f5-9c46-47cd7dd8060c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.693768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b3d0-1e5d-42f5-9c46-47cd7dd8060c None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.694885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032b3d0-1e5d-42f5-9c46-47cd7dd8060c None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:39.696367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032b3d0-1e5d-42f5-9c46-47cd7dd8060c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:39.698967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 220/444] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:39 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:39.699664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf589b1-9555-4cad-89f5-79ce79a0eecc None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.699944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf589b1-9555-4cad-89f5-79ce79a0eecc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.700212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf589b1-9555-4cad-89f5-79ce79a0eecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.718405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ebf589b1-9555-4cad-89f5-79ce79a0eecc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:39.719737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ebf589b1-9555-4cad-89f5-79ce79a0eecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:39.739709 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:39.784922 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00094fd-f3f6-402a-97ed-7d77c17f4cc3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.788796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 221/445] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:39.789620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf589b1-9555-4cad-89f5-79ce79a0eecc None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:39.828346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f6203-322d-4b84-b0ff-c6e87aab0615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:39.828985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f6203-322d-4b84-b0ff-c6e87aab0615 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.829259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f6203-322d-4b84-b0ff-c6e87aab0615 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.829527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f6203-322d-4b84-b0ff-c6e87aab0615 None admin] PATH_INFO: `/v3/projects/f618dc420f704f2c8a85cfe182a07300` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.830638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f6203-322d-4b84-b0ff-c6e87aab0615 None admin] RBAC: Authorizing `identity:get_project(project_id=f618dc420f704f2c8a85cfe182a07300)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:39.832100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f6203-322d-4b84-b0ff-c6e87aab0615 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:39.833872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 225/446] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:39 2022] GET /identity/v3/projects/f618dc420f704f2c8a85cfe182a07300 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:39.839450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f6203-322d-4b84-b0ff-c6e87aab0615 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:39.883053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:39.900124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf589b1-9555-4cad-89f5-79ce79a0eecc None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.944036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:39.944642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:39.944919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:39.945188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:39.945904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] RBAC: Adding query filter params (group.id=8c8057e0f50d4775902e859951a8277c, role.id=95375b0480bf4855a3ce51d1985c9b57, scope.project.id=f618dc420f704f2c8a85cfe182a07300) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:39.946328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:39.948270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:39.955487 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 222/447] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:39 2022] GET /identity/v3/role_assignments?group.id=8c8057e0f50d4775902e859951a8277c&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=95375b0480bf4855a3ce51d1985c9b57 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:39.958729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f6203-322d-4b84-b0ff-c6e87aab0615 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:39.999717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0698ee-404e-4e72-a450-30636e8499fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:40.000295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0698ee-404e-4e72-a450-30636e8499fb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:40.000569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0698ee-404e-4e72-a450-30636e8499fb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:40.000837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0698ee-404e-4e72-a450-30636e8499fb None admin] PATH_INFO: `/v3/groups/8c8057e0f50d4775902e859951a8277c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:40.002043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0698ee-404e-4e72-a450-30636e8499fb None admin] RBAC: Authorizing `identity:get_group(group_id=8c8057e0f50d4775902e859951a8277c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:40.003766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0698ee-404e-4e72-a450-30636e8499fb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:40.005649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 226/448] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:39 2022] GET /identity/v3/groups/8c8057e0f50d4775902e859951a8277c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:40.012637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:40.036846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:40.066486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0698ee-404e-4e72-a450-30636e8499fb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:40.136019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639fc3e-a012-466d-bb63-26e082ab1990 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:40.174696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:40.175307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:40.175586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:40.175848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:40.176574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] RBAC: Adding query filter params (group.id=8c8057e0f50d4775902e859951a8277c, role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=f618dc420f704f2c8a85cfe182a07300) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:40.176868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:40.178313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:40.187403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 223/449] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:40 2022] GET /identity/v3/role_assignments?group.id=8c8057e0f50d4775902e859951a8277c&scope.project.id=f618dc420f704f2c8a85cfe182a07300&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:40.188549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0698ee-404e-4e72-a450-30636e8499fb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:40.197297 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:40.228112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03d723-2e88-4a26-9004-d85664e15f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:40.229835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03d723-2e88-4a26-9004-d85664e15f91 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:40.230114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03d723-2e88-4a26-9004-d85664e15f91 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:40.230385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03d723-2e88-4a26-9004-d85664e15f91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:40.243999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e03d723-2e88-4a26-9004-d85664e15f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-5e03d723-2e88-4a26-9004-d85664e15f91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:40.250212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:40.251333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:40.252251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 227/450] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:40.257983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:40.356368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:40.375344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e5ce3-8ba1-42cb-bc26-705d998bd07f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:40.411594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:40.423859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:40.423859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:40.423859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:40.423859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:40.423859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:40.423859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:40.424580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 224/451] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:40.430032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03d723-2e88-4a26-9004-d85664e15f91 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:40.537116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03d723-2e88-4a26-9004-d85664e15f91 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:40.570958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456edd-acfd-4c6b-ad85-89d8e5ec8792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:40.571570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456edd-acfd-4c6b-ad85-89d8e5ec8792 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:40.571842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456edd-acfd-4c6b-ad85-89d8e5ec8792 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:40.572103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456edd-acfd-4c6b-ad85-89d8e5ec8792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:40.605339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:40.640165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be456edd-acfd-4c6b-ad85-89d8e5ec8792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:40.641629 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be456edd-acfd-4c6b-ad85-89d8e5ec8792 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:40.736385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 228/452] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:40.742541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:40.843303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071d14-eb10-4fea-9530-669d09305e56 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:40.878858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0db42-6a0b-4c7f-af56-d00620c401d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:40.879511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0db42-6a0b-4c7f-af56-d00620c401d7 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:40.879789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0db42-6a0b-4c7f-af56-d00620c401d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:40.880064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0db42-6a0b-4c7f-af56-d00620c401d7 None admin] PATH_INFO: `/v3/projects/27572d94c00243fdbbfd37ae021fbe77` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:40.881201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0db42-6a0b-4c7f-af56-d00620c401d7 None admin] RBAC: Authorizing `identity:get_project(project_id=27572d94c00243fdbbfd37ae021fbe77)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:40.882668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0db42-6a0b-4c7f-af56-d00620c401d7 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:40.884319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 225/453] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:35:40 2022] GET /identity/v3/projects/27572d94c00243fdbbfd37ae021fbe77 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:40.890964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456edd-acfd-4c6b-ad85-89d8e5ec8792 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:40.956080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:40.990842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456edd-acfd-4c6b-ad85-89d8e5ec8792 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:41.024020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e918a-78ae-4dc2-a5e4-44b0847344f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:41.024642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e918a-78ae-4dc2-a5e4-44b0847344f2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:41.024918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e918a-78ae-4dc2-a5e4-44b0847344f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:41.025189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e918a-78ae-4dc2-a5e4-44b0847344f2 None admin] PATH_INFO: `/v3/groups/09804aa6302d4ed182f5b3eb519232ad` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:41.026326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e918a-78ae-4dc2-a5e4-44b0847344f2 None admin] RBAC: Authorizing `identity:get_group(group_id=09804aa6302d4ed182f5b3eb519232ad)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:41.027782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e918a-78ae-4dc2-a5e4-44b0847344f2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:41.029341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 229/454] 10.222.0.94 () {56 vars in 1180 bytes} [Fri Aug 26 15:35:40 2022] GET /identity/v3/groups/09804aa6302d4ed182f5b3eb519232ad => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:41.035452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0db42-6a0b-4c7f-af56-d00620c401d7 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:41.051460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:41.136974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0db42-6a0b-4c7f-af56-d00620c401d7 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:41.170549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805232a-b477-4102-a0be-3e48272353d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:41.171172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805232a-b477-4102-a0be-3e48272353d5 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:41.171445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805232a-b477-4102-a0be-3e48272353d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:41.171715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805232a-b477-4102-a0be-3e48272353d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:41.172432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805232a-b477-4102-a0be-3e48272353d5 None admin] RBAC: Adding query filter params (group.id=09804aa6302d4ed182f5b3eb519232ad, role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=27572d94c00243fdbbfd37ae021fbe77) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:41.172759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805232a-b477-4102-a0be-3e48272353d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:41.174187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805232a-b477-4102-a0be-3e48272353d5 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:41.179787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 226/455] 10.222.0.94 () {56 vars in 1376 bytes} [Fri Aug 26 15:35:41 2022] GET /identity/v3/role_assignments?group.id=09804aa6302d4ed182f5b3eb519232ad&scope.project.id=27572d94c00243fdbbfd37ae021fbe77&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:43.692515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175487c-3b64-4f01-9265-b91d9c6f6caa None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:43.692515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175487c-3b64-4f01-9265-b91d9c6f6caa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:43.692515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175487c-3b64-4f01-9265-b91d9c6f6caa None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:43.693623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 230/456] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:43.697834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd273d3-1bc5-4bc1-86b5-16207ca32312 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:43.698804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd273d3-1bc5-4bc1-86b5-16207ca32312 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:43.699323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd273d3-1bc5-4bc1-86b5-16207ca32312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:43.714520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224343a-07bd-4512-b234-a351ece2b6d1 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:43.715180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224343a-07bd-4512-b234-a351ece2b6d1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:43.715646 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224343a-07bd-4512-b234-a351ece2b6d1 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:43.717164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 231/457] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:35:43.722424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa207a-2136-4ab0-b57a-67a7c951b927 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:43.722950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa207a-2136-4ab0-b57a-67a7c951b927 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:43.723425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa207a-2136-4ab0-b57a-67a7c951b927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:43.737309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6fd273d3-1bc5-4bc1-86b5-16207ca32312 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:43.743495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6fd273d3-1bc5-4bc1-86b5-16207ca32312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:43.760735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-58fa207a-2136-4ab0-b57a-67a7c951b927 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:43.762248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-58fa207a-2136-4ab0-b57a-67a7c951b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:43.844717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 227/458] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:43.905743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 232/459] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:44.120372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd273d3-1bc5-4bc1-86b5-16207ca32312 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:44.128885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:44.152131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512db266-7ef5-4e5b-a9cd-93cc1718175a None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.153043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512db266-7ef5-4e5b-a9cd-93cc1718175a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.153507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512db266-7ef5-4e5b-a9cd-93cc1718175a None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.156111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 233/460] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:44.162026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53641ad2-98ba-48f4-b31e-35bd13f72e7d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.162313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53641ad2-98ba-48f4-b31e-35bd13f72e7d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.162583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53641ad2-98ba-48f4-b31e-35bd13f72e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.199930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-53641ad2-98ba-48f4-b31e-35bd13f72e7d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:44.201126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-53641ad2-98ba-48f4-b31e-35bd13f72e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:44.244666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd273d3-1bc5-4bc1-86b5-16207ca32312 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:44.283463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:44.284283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.284562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.284830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.297715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:44.299877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:44.318261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:44.319249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:44.320071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:44.320071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:44.320071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:44.320071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 228/461] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:44.326064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:44.347514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:44.367366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 234/462] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:44.367366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53641ad2-98ba-48f4-b31e-35bd13f72e7d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:44.432300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:44.475586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53641ad2-98ba-48f4-b31e-35bd13f72e7d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:44.487227 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36623a8-e0f0-4ad6-8aa5-210fe01fabe4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:44.515475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca6264-0ce6-4464-a58a-78f89d32dae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:44.516123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca6264-0ce6-4464-a58a-78f89d32dae9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.516402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca6264-0ce6-4464-a58a-78f89d32dae9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.516671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca6264-0ce6-4464-a58a-78f89d32dae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.529804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca6264-0ce6-4464-a58a-78f89d32dae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:44.531407 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca6264-0ce6-4464-a58a-78f89d32dae9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:44.531859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec84e0-4173-49d8-8a61-b6388d1c3c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:44.532475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec84e0-4173-49d8-8a61-b6388d1c3c43 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.532752 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec84e0-4173-49d8-8a61-b6388d1c3c43 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.533019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec84e0-4173-49d8-8a61-b6388d1c3c43 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.533951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec84e0-4173-49d8-8a61-b6388d1c3c43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:44.534686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec84e0-4173-49d8-8a61-b6388d1c3c43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:44.536150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec84e0-4173-49d8-8a61-b6388d1c3c43 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-7aca6264-0ce6-4464-a58a-78f89d32dae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:44.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:44.549467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:44.550287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:44.550287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:44.550287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:44.550287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 235/463] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:44.550899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d9e4-d281-4beb-8ede-1e56f9c840f5 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.551086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d9e4-d281-4beb-8ede-1e56f9c840f5 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.551344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d9e4-d281-4beb-8ede-1e56f9c840f5 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.552560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 236/464] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:44.556735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 229/465] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:44.564143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5d9e4-d281-4beb-8ede-1e56f9c840f5 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:44.565919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09800-1c7b-4b57-9b0f-5967087705a0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.566203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09800-1c7b-4b57-9b0f-5967087705a0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.566472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09800-1c7b-4b57-9b0f-5967087705a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.589723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-eba09800-1c7b-4b57-9b0f-5967087705a0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:44.591341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-eba09800-1c7b-4b57-9b0f-5967087705a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:44.621091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:44.687164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 230/466] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:44.687969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba09800-1c7b-4b57-9b0f-5967087705a0 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:44.714467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5d9e4-d281-4beb-8ede-1e56f9c840f5 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:44.750214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:44.785580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:44.787223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.787980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.788919 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.790449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:44.791407 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:44.794598 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:44.804872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba09800-1c7b-4b57-9b0f-5967087705a0 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:44.827250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 237/467] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:44.829721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:44.841617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac17b28-506e-49d2-9a81-541e6412d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:44.842229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac17b28-506e-49d2-9a81-541e6412d49a None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:44.842496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac17b28-506e-49d2-9a81-541e6412d49a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:44.842946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac17b28-506e-49d2-9a81-541e6412d49a None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:44.843725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac17b28-506e-49d2-9a81-541e6412d49a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:44.845295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac17b28-506e-49d2-9a81-541e6412d49a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:44.958086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdab0e-7f35-4f2c-90dc-72b2ca089015 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:44.967618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:44.967924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 231/468] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:44 2022] POST /identity/v3/users => generated 248 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:44.970368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac17b28-506e-49d2-9a81-541e6412d49a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:45.002982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc8613-f4bf-4269-bc8f-6fd053cc14b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:45.003637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc8613-f4bf-4269-bc8f-6fd053cc14b9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.003917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc8613-f4bf-4269-bc8f-6fd053cc14b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.004305 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc8613-f4bf-4269-bc8f-6fd053cc14b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.016841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc8613-f4bf-4269-bc8f-6fd053cc14b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:45.018545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc8613-f4bf-4269-bc8f-6fd053cc14b9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-4ddc8613-f4bf-4269-bc8f-6fd053cc14b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:45.048440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:45.049409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:45.052235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:45.052235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:45.052235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:45.052235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 238/469] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:45.052235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911f953-0d43-4741-881d-74153c4a4228 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.052235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911f953-0d43-4741-881d-74153c4a4228 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.052893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911f953-0d43-4741-881d-74153c4a4228 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.056712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 239/470] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:45.057881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911f953-0d43-4741-881d-74153c4a4228 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:45.123324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac17b28-506e-49d2-9a81-541e6412d49a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:45.127215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:45.168679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb57881-b17c-4d4e-a57e-8022e1da816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:45.169305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb57881-b17c-4d4e-a57e-8022e1da816d None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.169584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb57881-b17c-4d4e-a57e-8022e1da816d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.169860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb57881-b17c-4d4e-a57e-8022e1da816d None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.170518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb57881-b17c-4d4e-a57e-8022e1da816d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:45.172050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb57881-b17c-4d4e-a57e-8022e1da816d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:45.193538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911f953-0d43-4741-881d-74153c4a4228 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:45.228155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4eab88-af2f-4220-9096-0466cab2875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:45.229420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eab88-af2f-4220-9096-0466cab2875c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.229732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eab88-af2f-4220-9096-0466cab2875c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.230076 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4eab88-af2f-4220-9096-0466cab2875c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.240946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4eab88-af2f-4220-9096-0466cab2875c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:45.242694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4eab88-af2f-4220-9096-0466cab2875c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:45.255966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 232/471] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:44 2022] POST /identity/v3/users => generated 246 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:45.256812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb57881-b17c-4d4e-a57e-8022e1da816d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-2a4eab88-af2f-4220-9096-0466cab2875c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:45.257815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:45.258643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:45.259813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:45.259813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:45.259813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:45.271088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 240/472] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:45.271088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b874-cbb4-46f8-8775-a60d50317d53 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.271088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b874-cbb4-46f8-8775-a60d50317d53 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.271088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b874-cbb4-46f8-8775-a60d50317d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.295924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2a00b874-cbb4-46f8-8775-a60d50317d53 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:45.297673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2a00b874-cbb4-46f8-8775-a60d50317d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:45.301862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:45.388415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 241/473] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:45.389641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00b874-cbb4-46f8-8775-a60d50317d53 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:45.423201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb57881-b17c-4d4e-a57e-8022e1da816d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:45.469578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:45.470191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.470471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.470752 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.471833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:45.472179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:45.474050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:45.491943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 233/474] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:45.497998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:45.524047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00b874-cbb4-46f8-8775-a60d50317d53 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:45.529066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:45.581603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:45.582655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.583196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.583628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.587697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:45.587697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:45.587697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:45.597093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:45.601263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 242/475] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:45.608268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:45.626216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486b1e4-06e4-455f-acab-f77932fc24f7 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:45.679363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98b38f-84bf-448a-877f-092e9da1baf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:45.680491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b38f-84bf-448a-877f-092e9da1baf4 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.680935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b38f-84bf-448a-877f-092e9da1baf4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.682415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98b38f-84bf-448a-877f-092e9da1baf4 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.683504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98b38f-84bf-448a-877f-092e9da1baf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:45.685744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98b38f-84bf-448a-877f-092e9da1baf4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:45.757460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a636098-b767-4499-9ff6-9251fa2830da None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:45.781132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fc559-7e13-4353-8134-a06a60fc1cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:45.781743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fc559-7e13-4353-8134-a06a60fc1cf8 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:45.782021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fc559-7e13-4353-8134-a06a60fc1cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:45.782353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fc559-7e13-4353-8134-a06a60fc1cf8 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:45.783014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fc559-7e13-4353-8134-a06a60fc1cf8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:45.784434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fc559-7e13-4353-8134-a06a60fc1cf8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:45.863804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 234/476] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:45 2022] POST /identity/v3/users => generated 248 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:35:45.865259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98b38f-84bf-448a-877f-092e9da1baf4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:45.901101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 243/477] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:45 2022] POST /identity/v3/users => generated 249 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:35:45.962371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:46.004589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98b38f-84bf-448a-877f-092e9da1baf4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:46.054360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:46.054938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:46.055234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:46.055510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:46.056212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:46.058303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:46.107951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.sql.core [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 15:35:46.107951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 15:35:46.107951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [parameters: {'id': '1c4bdaf514ad45e59b7a79ca0e95af05', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 15:35:46.107951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88509) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application result.read() Aug 26 15:35:46.125809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return ref.to_dict() Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:35:46.126747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application yield resource Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise self.value Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application session.commit() Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 15:35:46.128144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self.session.flush() Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application compat.raise_( Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 15:35:46.129270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:35:46.130125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application result.read() Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:35:46.130913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [parameters: {'id': '1c4bdaf514ad45e59b7a79ca0e95af05', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:46.131782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 15:35:46.134403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:46.137296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 235/478] 10.222.0.94 () {60 vars in 1131 bytes} [Fri Aug 26 15:35:45 2022] POST /identity/v3/roles => generated 144 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:46.137439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fc559-7e13-4353-8134-a06a60fc1cf8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:46.219293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:46.272359 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fc559-7e13-4353-8134-a06a60fc1cf8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:46.310698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03196150-ab3c-4fe7-bc91-625522a4a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:46.311291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03196150-ab3c-4fe7-bc91-625522a4a610 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:46.311668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03196150-ab3c-4fe7-bc91-625522a4a610 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:46.311941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03196150-ab3c-4fe7-bc91-625522a4a610 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:46.324285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03196150-ab3c-4fe7-bc91-625522a4a610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-03196150-ab3c-4fe7-bc91-625522a4a610 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:46.329489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:46.330374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:46.331176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 244/479] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:35:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:46.338045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:46.502694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce667b-630b-4909-b1fe-fdebfe2c815e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:46.545605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf5fc7-96f9-4293-a40b-22cbb72b2bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:46.546616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf5fc7-96f9-4293-a40b-22cbb72b2bd6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:46.547161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf5fc7-96f9-4293-a40b-22cbb72b2bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:46.547635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf5fc7-96f9-4293-a40b-22cbb72b2bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:46.548518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf5fc7-96f9-4293-a40b-22cbb72b2bd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:46.549020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf5fc7-96f9-4293-a40b-22cbb72b2bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:46.550592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf5fc7-96f9-4293-a40b-22cbb72b2bd6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:46.559734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 236/480] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:47.124079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54dca0-7768-44a7-8ca0-659a40a744d8 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:47.124592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54dca0-7768-44a7-8ca0-659a40a744d8 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:47.124592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54dca0-7768-44a7-8ca0-659a40a744d8 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:47.126227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 245/481] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:47.131443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c29628-3b93-4145-a696-fe3e87b0b59c None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:47.131880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c29628-3b93-4145-a696-fe3e87b0b59c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:47.132061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c29628-3b93-4145-a696-fe3e87b0b59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:47.141841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:47.156440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-72c29628-3b93-4145-a696-fe3e87b0b59c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:47.157635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-72c29628-3b93-4145-a696-fe3e87b0b59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:47.245264 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 237/482] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:47.605089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54dca0-7768-44a7-8ca0-659a40a744d8 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:47.744472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54dca0-7768-44a7-8ca0-659a40a744d8 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:47.766900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:47.796220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581297c-635f-4d62-9655-23beae452ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:47.796873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581297c-635f-4d62-9655-23beae452ea0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:47.797182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581297c-635f-4d62-9655-23beae452ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:47.797457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581297c-635f-4d62-9655-23beae452ea0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:47.812363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581297c-635f-4d62-9655-23beae452ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-a581297c-635f-4d62-9655-23beae452ea0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:47.817394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:47.818226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:47.819042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:47.824469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 246/483] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:47.832145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c29628-3b93-4145-a696-fe3e87b0b59c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:47.975756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c29628-3b93-4145-a696-fe3e87b0b59c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:48.023869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:48.034385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fccc5a-ce25-4642-81c1-f06f49df7915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:48.035326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fccc5a-ce25-4642-81c1-f06f49df7915 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.035764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fccc5a-ce25-4642-81c1-f06f49df7915 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.036189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fccc5a-ce25-4642-81c1-f06f49df7915 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.037783 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fccc5a-ce25-4642-81c1-f06f49df7915 None admin] RBAC: Adding query filter params (name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:48.037783 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fccc5a-ce25-4642-81c1-f06f49df7915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:48.039051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fccc5a-ce25-4642-81c1-f06f49df7915 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:48.050678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 238/484] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:48.055841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581297c-635f-4d62-9655-23beae452ea0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:48.082377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf7ce8-1a4b-4d2e-96fc-5b9e9ba8065a None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.083199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf7ce8-1a4b-4d2e-96fc-5b9e9ba8065a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.084967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf7ce8-1a4b-4d2e-96fc-5b9e9ba8065a None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.086507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 239/485] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:48.117009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf0ffc-806e-49c1-9755-9b6db5401ec7 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.117566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf0ffc-806e-49c1-9755-9b6db5401ec7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.118053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf0ffc-806e-49c1-9755-9b6db5401ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.154121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7fdf0ffc-806e-49c1-9755-9b6db5401ec7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:48.157590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7fdf0ffc-806e-49c1-9755-9b6db5401ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:48.220097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581297c-635f-4d62-9655-23beae452ea0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:48.266873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d57e5e-1a57-4481-ad66-8c39c6e59665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:48.267559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d57e5e-1a57-4481-ad66-8c39c6e59665 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.267842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d57e5e-1a57-4481-ad66-8c39c6e59665 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.268109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d57e5e-1a57-4481-ad66-8c39c6e59665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.286871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d57e5e-1a57-4481-ad66-8c39c6e59665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:48.288381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d57e5e-1a57-4481-ad66-8c39c6e59665 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:48.306288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 240/486] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-46d57e5e-1a57-4481-ad66-8c39c6e59665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:48.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:48.315753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:48.316973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:48.316973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:48.316973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:48.316973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 247/487] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:48.321973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf0ffc-806e-49c1-9755-9b6db5401ec7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:48.468236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf0ffc-806e-49c1-9755-9b6db5401ec7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:48.471177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95e18-2fd5-4248-b5e1-69adbc4f0338 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.471505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95e18-2fd5-4248-b5e1-69adbc4f0338 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.471782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c95e18-2fd5-4248-b5e1-69adbc4f0338 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.473129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 248/488] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:48.484812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedae9d0-f2cf-4cf9-8f46-ab5575fdc76b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.485118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedae9d0-f2cf-4cf9-8f46-ab5575fdc76b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.485345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedae9d0-f2cf-4cf9-8f46-ab5575fdc76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.495307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:48.508627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cedae9d0-f2cf-4cf9-8f46-ab5575fdc76b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:48.512537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86c0e0-76e5-4844-9bb3-eff7d1bd7bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:48.512537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c0e0-76e5-4844-9bb3-eff7d1bd7bfe None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.512537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c0e0-76e5-4844-9bb3-eff7d1bd7bfe None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.512936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cedae9d0-f2cf-4cf9-8f46-ab5575fdc76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:48.513076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c0e0-76e5-4844-9bb3-eff7d1bd7bfe None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.513076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c0e0-76e5-4844-9bb3-eff7d1bd7bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:48.513076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c0e0-76e5-4844-9bb3-eff7d1bd7bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:48.513412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c0e0-76e5-4844-9bb3-eff7d1bd7bfe None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:48.529589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 241/489] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:48.535400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c321-6d2f-4f92-b136-65e1106d4265 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.535731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c321-6d2f-4f92-b136-65e1106d4265 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.536011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748c321-6d2f-4f92-b136-65e1106d4265 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.537407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 242/490] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:48.538143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748c321-6d2f-4f92-b136-65e1106d4265 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:48.547257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:48.608399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 249/491] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:48.610055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9708a-1748-4524-be0f-41739cbfb818 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.610535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9708a-1748-4524-be0f-41739cbfb818 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.611048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9708a-1748-4524-be0f-41739cbfb818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.632608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-07e9708a-1748-4524-be0f-41739cbfb818 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:48.634300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-07e9708a-1748-4524-be0f-41739cbfb818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:48.669535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748c321-6d2f-4f92-b136-65e1106d4265 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:48.709659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:48.710316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.710588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.710851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.719619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:48.719949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 250/492] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:48.721072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9708a-1748-4524-be0f-41739cbfb818 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:48.721177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:48.735643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:48.736601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:48.737482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 243/493] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:48.742785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:48.793355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:48.846104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:48.853103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779eacc-bf55-4a96-b808-e29a80c2f3c1 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:48.863264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9708a-1748-4524-be0f-41739cbfb818 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:48.891767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:48.892505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.893182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.893182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.894187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:48.894551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:48.896532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:48.920000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67a771-3fa2-463f-9238-666b4faefc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:48.920635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67a771-3fa2-463f-9238-666b4faefc11 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:48.920952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67a771-3fa2-463f-9238-666b4faefc11 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:48.921214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67a771-3fa2-463f-9238-666b4faefc11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:48.934444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67a771-3fa2-463f-9238-666b4faefc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:48.939992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 244/494] 10.222.0.94 () {56 vars in 1137 bytes} [Fri Aug 26 15:35:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-6e67a771-3fa2-463f-9238-666b4faefc11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:48.940362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:48.941295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:48.942489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:48.944082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:48.951254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 251/495] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:48.951254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67a771-3fa2-463f-9238-666b4faefc11 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.076100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7687ef-d7bf-4a0c-b39c-874b3fe5db41 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.078188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:49.081967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67a771-3fa2-463f-9238-666b4faefc11 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.115163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb5b4d-c589-4d90-ba00-db62d8c819f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.115801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb5b4d-c589-4d90-ba00-db62d8c819f8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.116074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb5b4d-c589-4d90-ba00-db62d8c819f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.116586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb5b4d-c589-4d90-ba00-db62d8c819f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.119943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3c3cc-e84f-46a1-ac5e-354ce6313936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.120557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3c3cc-e84f-46a1-ac5e-354ce6313936 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.120836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3c3cc-e84f-46a1-ac5e-354ce6313936 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.121094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3c3cc-e84f-46a1-ac5e-354ce6313936 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.130502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb5b4d-c589-4d90-ba00-db62d8c819f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-34bb5b4d-c589-4d90-ba00-db62d8c819f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:49.136005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:49.136872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:49.137637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 245/496] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:49.138274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb5b4d-c589-4d90-ba00-db62d8c819f8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.144653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:49.199837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3c3cc-e84f-46a1-ac5e-354ce6313936 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:49.201373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3c3cc-e84f-46a1-ac5e-354ce6313936 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:49.251307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb5b4d-c589-4d90-ba00-db62d8c819f8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.284248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.284848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.285119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.285386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.286038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] RBAC: Adding query filter params (name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:49.286330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:49.287739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:49.296415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 246/497] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:49.297035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.310177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 252/498] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:49.311061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3c3cc-e84f-46a1-ac5e-354ce6313936 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.337565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:49.361477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:49.414147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b6c84-52b1-401c-8fc1-fad5257b22ff None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.416201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3c3cc-e84f-46a1-ac5e-354ce6313936 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.450650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1e484-99b9-44df-9cca-105546e994c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.451317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1e484-99b9-44df-9cca-105546e994c0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.451591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1e484-99b9-44df-9cca-105546e994c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.451899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1e484-99b9-44df-9cca-105546e994c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.453362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa13936-c168-45f1-90f8-19e07146e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.454035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa13936-c168-45f1-90f8-19e07146e39f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.454338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa13936-c168-45f1-90f8-19e07146e39f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.454589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa13936-c168-45f1-90f8-19e07146e39f None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.455281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa13936-c168-45f1-90f8-19e07146e39f None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:49.455584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa13936-c168-45f1-90f8-19e07146e39f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:49.456979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa13936-c168-45f1-90f8-19e07146e39f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:49.462473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1e484-99b9-44df-9cca-105546e994c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:49.465737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 253/499] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-87e1e484-99b9-44df-9cca-105546e994c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:49.467738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:49.468569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:49.469679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:49.470611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04a89c-75d5-43c1-84cb-6c4e16d06395 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.470611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04a89c-75d5-43c1-84cb-6c4e16d06395 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.470829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 247/500] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:49.470984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04a89c-75d5-43c1-84cb-6c4e16d06395 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.470984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 254/501] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:49.471173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04a89c-75d5-43c1-84cb-6c4e16d06395 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.471918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1e484-99b9-44df-9cca-105546e994c0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.563479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:49.580017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1e484-99b9-44df-9cca-105546e994c0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.582435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04a89c-75d5-43c1-84cb-6c4e16d06395 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.593918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:49.617416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.617964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.618102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.618308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.618583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.618669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.618903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.619172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.629032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:49.629958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:49.630467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:49.631442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:49.645558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:49.646437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:49.647529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:49.647529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:49.647529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:49.647688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:49.648452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:49.649338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:49.649338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:49.649338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:49.649521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 248/502] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:49.649588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 255/503] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:49.649651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.649723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.752358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0ea3e-4113-4e5e-bf7c-c561ecf1645d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.753519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9b1a1-6117-4b71-9043-8ff0774ea864 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.786329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615f15e-b679-4f62-a249-5ca2ecbe32e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.786979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615f15e-b679-4f62-a249-5ca2ecbe32e0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.787253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615f15e-b679-4f62-a249-5ca2ecbe32e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.787522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615f15e-b679-4f62-a249-5ca2ecbe32e0 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.788044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.788178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615f15e-b679-4f62-a249-5ca2ecbe32e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:49.788475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615f15e-b679-4f62-a249-5ca2ecbe32e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:49.788672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.788948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.789214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.789841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615f15e-b679-4f62-a249-5ca2ecbe32e0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:49.792575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:49.798800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 256/504] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:49.800159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af675f42-8dcc-4dc1-beb8-d50e3f36a6ac None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.800388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af675f42-8dcc-4dc1-beb8-d50e3f36a6ac None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.800657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af675f42-8dcc-4dc1-beb8-d50e3f36a6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.801953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:49.803480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:49.817584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:49.818664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:49.819839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:49.819839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:49.819839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:49.819997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-af675f42-8dcc-4dc1-beb8-d50e3f36a6ac None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:49.820066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 249/505] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:49.820066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.820187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-af675f42-8dcc-4dc1-beb8-d50e3f36a6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:49.838304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:49.889055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 257/506] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:49.890004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af675f42-8dcc-4dc1-beb8-d50e3f36a6ac None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:49.949147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807e7fa-cd1b-45d3-a173-907fa184d07c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:49.987340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:49.987950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:49.988226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:49.988494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:49.989144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:49.989443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:49.990807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.007952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 250/507] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:50.007952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.019590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af675f42-8dcc-4dc1-beb8-d50e3f36a6ac None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.061320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.061968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.062251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.062516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.063215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:50.063517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.063748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:50.064884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.077909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 258/508] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:50.079075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.098580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:50.127195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33fc7c7-5191-4b70-aad0-b8045bfa61d4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.172609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.173159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.173432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.173696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.185691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.188394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.195839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be861bed-92c7-443c-bfab-19ceecb8f275 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:50.202192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:50.203597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:50.204413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:50.204413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:50.204413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:50.204413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 251/509] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:50.204413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.239726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.241006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.241721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.242157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.243175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:50.243752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.245950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.263599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 259/510] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:50.263599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.311897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:50.315582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1447397-b810-4f8d-86a5-598d490647fc None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.348330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:50.352387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.353014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.353937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.354203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.362324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.364022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.376010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108f53-9073-4fdb-94a9-097e68221299 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:50.377263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:50.378201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:50.379122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 252/511] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:50.379665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.419841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.420670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.420827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.420975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.428593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.430107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:50.444612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:50.445510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:50.446334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 260/512] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:50.447715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.496047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6147a-1c14-42ea-b25f-a71d365d0b35 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.532345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.532968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.533251 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.533524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.546223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.548841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:50.566698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:50.567986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:50.568883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:50.568883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:50.568883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:50.568883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 253/513] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:50.568883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.573056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28804ce-23e8-4a35-ba6c-4d7d250147c8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.589509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:50.611081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.611870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.612210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.612540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.613258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:50.613644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.615766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.618234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:50.630171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 261/514] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:50.631102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.693278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d20dd-09eb-4eda-ac04-9d2fea57b63d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.733009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.733700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.733985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.734255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.739462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9568a-9f0a-4ee8-80e4-9018134a7972 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.742546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.744078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:50.756941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:50.757871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:50.758757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 254/515] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:50.760592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.779568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.780171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.780440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.780702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.781427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:50.781717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.783766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.822433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 262/516] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:50.824305 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.839260 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:50.877137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d647fbc-276d-4e43-9a7a-82803532a5bf None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.883287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:50.918143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.918752 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.919061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.919331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.920017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:50.920314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.921731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.935379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2ab1e-27ad-4407-bb7e-c71d04ff651d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:50.939243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 255/517] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:50.940058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:50.969892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:50.970576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:50.970848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:50.971135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:50.971778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:50.972068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:50.973441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:50.985468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 263/518] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:50.986281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:51.050857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6396ae-507b-4132-80d4-20f9ee2b42d3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:51.081571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:51.086474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:51.087145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:51.088105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:51.088105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:51.090735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a272293-dc78-4208-91f2-69924811be69 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:51.095242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:51.096739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:51.104579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:51.108871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:51.109732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:51.110643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 256/519] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:51.111809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:51.128122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:51.128738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:51.129012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:51.129277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:51.130008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:51.130309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:51.131779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:51.151261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 264/520] 10.222.0.94 () {56 vars in 1139 bytes} [Fri Aug 26 15:35:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:51.152105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:51.218682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9675c8b-4ced-42b3-99b5-cb4082602a15 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:51.253850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8c17a-aeec-4fa7-a970-8aa043f2010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:51.254060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e68f1-0be5-4c5c-9c0d-258cdde8285e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:51.254369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c17a-aeec-4fa7-a970-8aa043f2010e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:51.254647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c17a-aeec-4fa7-a970-8aa043f2010e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:51.254960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8c17a-aeec-4fa7-a970-8aa043f2010e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:51.288211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:51.288813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:51.289086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:51.289352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:51.290084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=23ddf8f600454da7ae9b715cd4cbd186) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:51.290431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:51.291881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:51.297421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 265/521] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:51 2022] GET /identity/v3/role_assignments?user.id=23ddf8f600454da7ae9b715cd4cbd186&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:51.298188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:51.318849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:51.331397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c17a-aeec-4fa7-a970-8aa043f2010e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:51.338445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8c17a-aeec-4fa7-a970-8aa043f2010e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:51.339768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:51.419013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec55e-1583-43de-b800-1cba1f104283 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:51.450477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 257/522] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:51.451833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8c17a-aeec-4fa7-a970-8aa043f2010e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:51.463320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d9695-f7ae-4eea-9fb8-fbefa14f3c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:51.464468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d9695-f7ae-4eea-9fb8-fbefa14f3c60 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:51.464990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d9695-f7ae-4eea-9fb8-fbefa14f3c60 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:51.465399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d9695-f7ae-4eea-9fb8-fbefa14f3c60 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:51.466189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d9695-f7ae-4eea-9fb8-fbefa14f3c60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:51.467892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d9695-f7ae-4eea-9fb8-fbefa14f3c60 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:51.552585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 266/523] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:35:51 2022] POST /identity/v3/users => generated 247 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:35:51.553483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d9695-f7ae-4eea-9fb8-fbefa14f3c60 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:51.577947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:51.585130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8c17a-aeec-4fa7-a970-8aa043f2010e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:51.636749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:51.638073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:51.638436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:51.638781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:51.640189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:51.640560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:51.642449 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:51.670656 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 258/524] 10.222.0.94 () {56 vars in 1137 bytes} [Fri Aug 26 15:35:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:51.672311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:51.674987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:51.686396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d9695-f7ae-4eea-9fb8-fbefa14f3c60 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:51.729919 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e76c4-24a2-44d7-aace-2ea5af4066bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:51.730539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e76c4-24a2-44d7-aace-2ea5af4066bd None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:51.730833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e76c4-24a2-44d7-aace-2ea5af4066bd None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:51.731124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e76c4-24a2-44d7-aace-2ea5af4066bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:51.812267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47193c69-47b2-4e6c-a869-4006058fcc0b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:51.822866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e76c4-24a2-44d7-aace-2ea5af4066bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:51.824388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e76c4-24a2-44d7-aace-2ea5af4066bd None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:51.853053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:51.854004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:51.854380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:51.854655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:51.866224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:51.867984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:51.886177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:51.887383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:51.888234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:51.888234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:51.888234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:51.888234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 259/525] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:51.888234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:51.895828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:51.940851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:51.959204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 267/526] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:51.960926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e76c4-24a2-44d7-aace-2ea5af4066bd None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.016206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052d479-46c3-4bde-89bd-e0a3eaac4578 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.054081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c7423-1631-40a9-a735-d042a99d7034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.054688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7423-1631-40a9-a735-d042a99d7034 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.054985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7423-1631-40a9-a735-d042a99d7034 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.055259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7423-1631-40a9-a735-d042a99d7034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.109769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e76c4-24a2-44d7-aace-2ea5af4066bd None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.137287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c7423-1631-40a9-a735-d042a99d7034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.138770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c7423-1631-40a9-a735-d042a99d7034 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.141030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:52.166687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.168053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.168812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.169320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.170685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:52.171506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.173565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.192592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 268/527] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:52.194138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.257307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:52.269496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 260/528] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:52.275155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c7423-1631-40a9-a735-d042a99d7034 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.316881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4e228-7d14-4bfe-ac70-c52f173483b3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.353631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.354267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.354611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.354914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.366602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.368028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:52.385206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:52.386380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:52.387342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:52.387342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:52.387342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:52.390987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 269/529] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:52.391861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.419077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c7423-1631-40a9-a735-d042a99d7034 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.423580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:52.470818 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.472393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.472832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.473273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.483156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.485520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:52.502786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:52.503905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:52.505017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:52.505017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:52.505017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:52.505017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:52.505017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:52.505017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:52.505017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:52.505017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:52.506642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 261/530] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:52.507712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.520596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bcb75-0ebb-46ed-b73d-8d627ae10f13 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.523180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:52.567628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.568754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.569208 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.569954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.583311 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.585432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:52.605963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:52.607127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:52.607999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:52.607999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:52.607999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:52.610043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 270/531] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:52.611124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.633629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2ba48-7b63-42cc-bc4a-c25010c5d835 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.671571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.672270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.672661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.672927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.673633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:52.674025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.675776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.693557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 262/532] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:52.694618 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.721913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:52.725607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347679d-7c00-431f-84b3-59e7a6aefd58 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.762834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.763817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.764295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.764786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.765711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:52.766211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.767877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.789897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 271/533] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:52.791941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.808170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a327f4-ad59-4922-8f66-8b9c27b3f93b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.809347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:52.845451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.846028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.846365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.846637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.847309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:52.847603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.849269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.862579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 263/534] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:52.863549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.902951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7029-7d04-4e95-9f3c-4a354ce9942c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:52.939255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:52.940254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:52.940735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:52.941225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:52.949581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:52.951319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:52.956109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:52.964135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:52.969693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 272/535] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:52.970591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:52.982026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da548382-c241-46f7-9b00-f426034c9197 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:53.022393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:53.023077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.023362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.023749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.024446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=500d2f2bd83d499aa2814c6d7cc71795) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:53.024743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:53.027001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:53.033476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 264/536] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/role_assignments?user.id=500d2f2bd83d499aa2814c6d7cc71795&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:53.034383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:53.055444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:53.085293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1682ec-85ec-4e46-8587-0fc524ad8d66 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:53.132692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:53.133431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.133787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.134104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.154441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:53.155844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:53.176552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:53.177443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:53.178251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 273/537] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:53.189364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:53.217612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b8c39-1af5-49e8-b2cb-4443771aa44e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:53.266838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:53.267502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.268116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.268400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.269110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:53.269469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:53.271657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:53.300136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 265/538] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:53.303789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:53.322016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:53.384479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974edc-0d53-49dd-9e67-32acf6307487 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:53.436966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c5b00-300a-4bbd-ad73-5af8558066d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:53.437489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c5b00-300a-4bbd-ad73-5af8558066d0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.437767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c5b00-300a-4bbd-ad73-5af8558066d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.438036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c5b00-300a-4bbd-ad73-5af8558066d0 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.438731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c5b00-300a-4bbd-ad73-5af8558066d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:53.439084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c5b00-300a-4bbd-ad73-5af8558066d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:53.440547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c5b00-300a-4bbd-ad73-5af8558066d0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:53.442538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:53.456132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 274/539] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:53.457271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeefb5-d285-4999-b9db-742cdea89583 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.457555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeefb5-d285-4999-b9db-742cdea89583 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.457826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeefb5-d285-4999-b9db-742cdea89583 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.463074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 275/540] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:53.463074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeefb5-d285-4999-b9db-742cdea89583 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:53.478789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b628b-7f2d-47e9-a8e1-9c1d2a7b30e6 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:53.522496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cde7cd-371a-475c-935b-15642adf995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:53.523376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde7cd-371a-475c-935b-15642adf995e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.523792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde7cd-371a-475c-935b-15642adf995e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.524151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde7cd-371a-475c-935b-15642adf995e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.526318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cde7cd-371a-475c-935b-15642adf995e None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=b7ad758153334ec28940156d09495ae6) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:53.526724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cde7cd-371a-475c-935b-15642adf995e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:53.528780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cde7cd-371a-475c-935b-15642adf995e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:53.536665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 266/541] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:53 2022] GET /identity/v3/role_assignments?user.id=b7ad758153334ec28940156d09495ae6&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:53.538616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171379e3-93bc-45bc-a4ef-75c4f3dfbd1e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.538997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171379e3-93bc-45bc-a4ef-75c4f3dfbd1e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.539384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171379e3-93bc-45bc-a4ef-75c4f3dfbd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.580751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-171379e3-93bc-45bc-a4ef-75c4f3dfbd1e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:53.583430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-171379e3-93bc-45bc-a4ef-75c4f3dfbd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:53.663623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeefb5-d285-4999-b9db-742cdea89583 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:53.695953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:53.701677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 267/542] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:53.703517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674c2b4-fb29-4519-88ed-9ad174af0e46 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.704048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674c2b4-fb29-4519-88ed-9ad174af0e46 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.704560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674c2b4-fb29-4519-88ed-9ad174af0e46 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.707245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 268/543] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:53.710220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:53.710220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.710220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.710220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.712737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=58567e28a5f741eda021aecab43d6a63) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:53.712737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:53.712737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:53.713013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e69bcc-d7bc-4ce9-9689-6c51df51fcd3 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:53.713332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e69bcc-d7bc-4ce9-9689-6c51df51fcd3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:53.713744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e69bcc-d7bc-4ce9-9689-6c51df51fcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:53.719712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 276/544] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:53 2022] GET /identity/v3/role_assignments?user.id=58567e28a5f741eda021aecab43d6a63&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:53.735820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-55e69bcc-d7bc-4ce9-9689-6c51df51fcd3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:53.737321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-55e69bcc-d7bc-4ce9-9689-6c51df51fcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:53.823018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 269/545] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:35:54.091527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:54.165148 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e69bcc-d7bc-4ce9-9689-6c51df51fcd3 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:54.171310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:54.218706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8085f-0ad1-4b29-9475-c9feacb162a4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:54.277341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccaa66-c97d-44a3-bb53-3252e7a12a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:54.278413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccaa66-c97d-44a3-bb53-3252e7a12a50 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:54.280025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccaa66-c97d-44a3-bb53-3252e7a12a50 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:54.281198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccaa66-c97d-44a3-bb53-3252e7a12a50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:54.300353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccaa66-c97d-44a3-bb53-3252e7a12a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-e9ccaa66-c97d-44a3-bb53-3252e7a12a50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:54.305614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:54.307044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:54.308052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 277/546] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:54.315104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccaa66-c97d-44a3-bb53-3252e7a12a50 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:54.383972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e69bcc-d7bc-4ce9-9689-6c51df51fcd3 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:54.450549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:54.465114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccaa66-c97d-44a3-bb53-3252e7a12a50 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:54.471701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e9820-79a3-43c9-bfce-cbfea09d71f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:54.472649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9820-79a3-43c9-bfce-cbfea09d71f9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:54.473122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9820-79a3-43c9-bfce-cbfea09d71f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:54.473596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9820-79a3-43c9-bfce-cbfea09d71f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:54.484983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:54.499865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e9820-79a3-43c9-bfce-cbfea09d71f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-5d1e9820-79a3-43c9-bfce-cbfea09d71f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:54.505337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:54.506292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:54.507090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:54.510576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 270/547] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:54.516041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e9820-79a3-43c9-bfce-cbfea09d71f9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:54.520219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:54.520843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:54.521123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:54.521395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:54.522075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:54.522377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:54.523782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:54.533277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 278/548] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:54.538782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:54.710169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e9820-79a3-43c9-bfce-cbfea09d71f9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:54.737414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6828-8018-4d2c-b388-b8f2c0cbb75c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:54.764953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:54.765570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:54.765847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:54.766117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:54.766791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] RBAC: Adding query filter params (name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:54.771090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:54.771090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:54.785919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 271/549] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:35:54.792378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:54.813195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:54.813949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:54.814228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:54.814498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:54.828912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:54.830479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:54.846384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:54.847364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:54.848196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:54.848196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:54.848196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:54.848196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 279/550] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:35:54.855630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:54.913222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:54.918442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:54.971592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a69ab-aa61-4180-b47c-039404efc00c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:55.014705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:55.015675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.015957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.016230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.030242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:55.031791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:55.049410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:55.051573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:55.053087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:55.053087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:55.053087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:55.053087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 272/551] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:55.060194 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:55.092429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f785d12-b343-44e4-b418-e745c37685d2 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:55.175655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:55.176655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.177160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.177524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.178276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:55.178665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:55.183113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:55.201189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 280/552] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:55.211988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:55.226888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35f37-3f95-4067-9e94-6214b1373637 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:55.263382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:55.264001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.264462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.264741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.265395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:55.265696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:55.267334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:55.276569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:55.283606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 273/553] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:55.293598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:55.356719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:55.365479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ee2a-a7e5-4f12-bbf9-3a9b0e38ecd7 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:55.409901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ffe4a-9cd5-42d2-8f49-9b996d8b78ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:55.410269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66155b-4bab-4435-ab54-4bda09bc7d26 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:55.411142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ffe4a-9cd5-42d2-8f49-9b996d8b78ed None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.411623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ffe4a-9cd5-42d2-8f49-9b996d8b78ed None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.412374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ffe4a-9cd5-42d2-8f49-9b996d8b78ed None admin] PATH_INFO: `/v3/users/glance` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.424109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ffe4a-9cd5-42d2-8f49-9b996d8b78ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:55.426258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ffe4a-9cd5-42d2-8f49-9b996d8b78ed None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-db3ffe4a-9cd5-42d2-8f49-9b996d8b78ed None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:55.445273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:55.448354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 281/554] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebcf5a-8e49-4431-ab91-dde715fb978d None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.449262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebcf5a-8e49-4431-ab91-dde715fb978d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.450014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebcf5a-8e49-4431-ab91-dde715fb978d None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.450956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 282/555] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:55.452996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebcf5a-8e49-4431-ab91-dde715fb978d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:55.460665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9e876-a0c1-4cfe-9624-09df8dfa9ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:55.461452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9e876-a0c1-4cfe-9624-09df8dfa9ffc None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.461825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9e876-a0c1-4cfe-9624-09df8dfa9ffc None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.462171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9e876-a0c1-4cfe-9624-09df8dfa9ffc None admin] PATH_INFO: `/v3/users/swift` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.473913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9e876-a0c1-4cfe-9624-09df8dfa9ffc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:55.475819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9e876-a0c1-4cfe-9624-09df8dfa9ffc None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-98c9e876-a0c1-4cfe-9624-09df8dfa9ffc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:55.494673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:55.495782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:55.497190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 274/556] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:35:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:55.497761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75759c-b7e3-42b6-9591-b18fa2d22a8a None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.497761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75759c-b7e3-42b6-9591-b18fa2d22a8a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.497889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75759c-b7e3-42b6-9591-b18fa2d22a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.528160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3d75759c-b7e3-42b6-9591-b18fa2d22a8a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:55.529840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3d75759c-b7e3-42b6-9591-b18fa2d22a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:55.555336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:55.596220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebcf5a-8e49-4431-ab91-dde715fb978d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:55.629882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 275/557] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:55.631134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75759c-b7e3-42b6-9591-b18fa2d22a8a None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:55.657942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:55.659263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.660539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.661002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.662725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:55.663419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:55.666282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:55.713525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 283/558] 10.222.0.94 () {56 vars in 1137 bytes} [Fri Aug 26 15:35:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:55.723927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:55.749620 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:55.761667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75759c-b7e3-42b6-9591-b18fa2d22a8a None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:55.814030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:55.815021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.816068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.817410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.818392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:55.819363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:55.821203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:55.857530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 276/559] 10.222.0.94 () {56 vars in 1135 bytes} [Fri Aug 26 15:35:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:55.865033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:55.900440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c16c-1ba3-4869-b08b-f60b32b1969f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:55.912822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:55.962289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:55.963877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:55.964159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:55.964427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:55.978476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:55.980488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:55.997109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:55.998102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:55.999111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:55.999111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:55.999111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:55.999111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 284/560] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:55.999848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.028894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865cd18-df43-4d94-ac25-6570f9161b3e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.064162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c922e-da28-4e69-83ef-3596bbc97122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.064744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c922e-da28-4e69-83ef-3596bbc97122 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.065017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c922e-da28-4e69-83ef-3596bbc97122 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.065286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c922e-da28-4e69-83ef-3596bbc97122 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.065467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:56.105807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd858f-9cca-47ab-9042-fd0301f6cd95 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.136879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c922e-da28-4e69-83ef-3596bbc97122 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:56.138332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c922e-da28-4e69-83ef-3596bbc97122 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:56.139685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6c9f1-2b8f-431a-872d-607494f60a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.140310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c9f1-2b8f-431a-872d-607494f60a1b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.140576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c9f1-2b8f-431a-872d-607494f60a1b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.140843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c9f1-2b8f-431a-872d-607494f60a1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.151497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6c9f1-2b8f-431a-872d-607494f60a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:56.152340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8bf6c9f1-2b8f-431a-872d-607494f60a1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:56.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:56.157676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:56.158710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 285/561] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:56.159579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b89bf-55a6-4e3f-b223-72baa502550b None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.159579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b89bf-55a6-4e3f-b223-72baa502550b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.159740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b89bf-55a6-4e3f-b223-72baa502550b None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.161033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 286/562] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:56.161727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b89bf-55a6-4e3f-b223-72baa502550b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.243413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 277/563] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:35:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:35:56.244958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ff569-ccc5-43b8-a5a7-1afd691a9609 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.245240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ff569-ccc5-43b8-a5a7-1afd691a9609 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.245506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ff569-ccc5-43b8-a5a7-1afd691a9609 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.246890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 278/564] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:56.247474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ff569-ccc5-43b8-a5a7-1afd691a9609 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.268055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b89bf-55a6-4e3f-b223-72baa502550b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.291220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:56.302982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09a6f0-9a42-4fad-bbe3-36fd3579cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.303593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a6f0-9a42-4fad-bbe3-36fd3579cbbf None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.303869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a6f0-9a42-4fad-bbe3-36fd3579cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.304136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a6f0-9a42-4fad-bbe3-36fd3579cbbf None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.304777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09a6f0-9a42-4fad-bbe3-36fd3579cbbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:56.305076 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09a6f0-9a42-4fad-bbe3-36fd3579cbbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:56.307075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09a6f0-9a42-4fad-bbe3-36fd3579cbbf None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:56.318687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 287/565] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:56.319942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0211a-b166-415e-bf67-074de43d7344 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.320229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0211a-b166-415e-bf67-074de43d7344 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.320501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0211a-b166-415e-bf67-074de43d7344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.339272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-78f0211a-b166-415e-bf67-074de43d7344 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:56.340476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-78f0211a-b166-415e-bf67-074de43d7344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:56.353858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ff569-ccc5-43b8-a5a7-1afd691a9609 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.367864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:56.391020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.391684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.399611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.399611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.399611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] RBAC: Adding query filter params (name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:56.399611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:56.399611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:56.402926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 279/566] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:56.403771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.412802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 288/567] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:56.414176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e13f-0244-42c5-8dda-86b3db32b043 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.414460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e13f-0244-42c5-8dda-86b3db32b043 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.414723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e13f-0244-42c5-8dda-86b3db32b043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.440888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6762e13f-0244-42c5-8dda-86b3db32b043 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:56.442482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6762e13f-0244-42c5-8dda-86b3db32b043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:56.519292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c8fda-5fc9-4f81-a122-2e52055ace46 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.525945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 289/568] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:56.527294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762e13f-0244-42c5-8dda-86b3db32b043 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.572987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.573611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.573983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.574319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.575567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:56.590326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:56.593593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:56.612479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:56.613466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:56.614332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:56.614332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:56.614332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:56.619957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 280/569] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:56.620808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.646787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762e13f-0244-42c5-8dda-86b3db32b043 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.661528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:56.686161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.687232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.687701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.688181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.696411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:56.697950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:56.709333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:56.710226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:56.711107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 290/570] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:56.711580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.751188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a24a02-b0cd-4e2f-98be-98791ebf01f3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.787633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.788191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.788464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.788732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.801243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:56.802675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:56.817580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:56.818526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:56.819411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:56.819411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:56.819411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:56.819411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 281/571] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:56.819641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.823895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1cd8f-1879-4ab2-ac7e-3e124786b505 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.825632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:56.860357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.860966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.861244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.861514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.862156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:56.862459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:56.863853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:56.876482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 291/572] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:56.877299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.903388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:56.925435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3bb36-697a-4e9b-ad54-10154ad05cd3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:56.959551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612e1f0-09eb-46e5-bdca-edacd8c88d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:56.960212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612e1f0-09eb-46e5-bdca-edacd8c88d5c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:56.960488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612e1f0-09eb-46e5-bdca-edacd8c88d5c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:56.960755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612e1f0-09eb-46e5-bdca-edacd8c88d5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:56.971807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612e1f0-09eb-46e5-bdca-edacd8c88d5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-f612e1f0-09eb-46e5-bdca-edacd8c88d5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:56.977022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:56.978464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:56.979558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 282/573] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:56.980850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612e1f0-09eb-46e5-bdca-edacd8c88d5c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:56.980921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4515057-9666-4dcb-8779-8c860c277d44 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.020694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.021300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.021582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.021857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.022551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:57.022853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.024351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.043708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 292/574] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:57.043708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.064735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:57.086985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612e1f0-09eb-46e5-bdca-edacd8c88d5c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.123365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97eda-4354-4df6-9502-0d1b4e7dcefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.124002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97eda-4354-4df6-9502-0d1b4e7dcefe None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.124276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97eda-4354-4df6-9502-0d1b4e7dcefe None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.124547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97eda-4354-4df6-9502-0d1b4e7dcefe None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.131410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:57.135495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c97eda-4354-4df6-9502-0d1b4e7dcefe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-16c97eda-4354-4df6-9502-0d1b4e7dcefe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:35:57.140770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:57.147436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:57.148284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 283/575] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:57.148963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2af2-f7cc-4ba0-a09b-af069ea2d859 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.149038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97eda-4354-4df6-9502-0d1b4e7dcefe None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.182950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.183586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.183862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.184127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.184772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:57.185063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.186424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.197894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 293/576] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:57.198698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.247443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97eda-4354-4df6-9502-0d1b4e7dcefe None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.282344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:57.282713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.283375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.283646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.283910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.290524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.292123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.300162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd09fb-1285-4578-bf77-e8ffccffc8a1 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:57.303343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:57.304216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:57.305120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 284/577] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:57.305595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.335904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.336517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.336791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.337061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.337927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:57.338203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.339616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.347625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 294/578] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:57.348423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.364437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:57.408582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43adb04b-8b3b-42b3-a3ab-53d984267c01 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.444797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127fd25-cb4c-4ba7-8ff6-7bb48d2a3be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.445461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127fd25-cb4c-4ba7-8ff6-7bb48d2a3be4 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.445735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127fd25-cb4c-4ba7-8ff6-7bb48d2a3be4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.446006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127fd25-cb4c-4ba7-8ff6-7bb48d2a3be4 None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/23ddf8f600454da7ae9b715cd4cbd186/roles/b8b6b507d5284b84a25895b280b89cbd` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.452569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc09d48-d7c3-4614-aaaf-9389384d3b89 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.461362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127fd25-cb4c-4ba7-8ff6-7bb48d2a3be4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=23ddf8f600454da7ae9b715cd4cbd186, role_id=b8b6b507d5284b84a25895b280b89cbd)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.463071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127fd25-cb4c-4ba7-8ff6-7bb48d2a3be4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.490238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.490865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.491431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.491692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.492342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:57.492672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.494624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.502666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 295/579] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:57.503477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.514072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:57.558661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 285/580] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:35:57 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/23ddf8f600454da7ae9b715cd4cbd186/roles/b8b6b507d5284b84a25895b280b89cbd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:57.559535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127fd25-cb4c-4ba7-8ff6-7bb48d2a3be4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.620899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60805e-3218-4277-8510-f96301f46300 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.662257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.662908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.663212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.663481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.673336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.675118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:57.689219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:57.690351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:35:57.691300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:57.695408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 296/581] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:57.696396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.700203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127fd25-cb4c-4ba7-8ff6-7bb48d2a3be4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.713939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:57.746033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.747018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.747763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.748177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.749316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:57.749776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.751361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.765310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 286/582] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:57.766350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.774046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:57.810666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85eb02-0e32-4504-86be-7d2fe3fe38b2 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.846902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.847600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.847838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.848291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.861249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.862708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:57.878447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:57.879469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:57.880358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:57.880358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:57.880358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:57.880358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 297/583] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:57.880358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.889888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edd271-5038-42b8-84d1-b57e18cc4602 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:57.926652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:57.927379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:57.927658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:57.927923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:57.938553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:57.940037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:57.954650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:57.955988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:57.956870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:57.956870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:57.956870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:57.960387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 287/584] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:57.961237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:57.985211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:58.000627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51675d7-398b-4d10-a1d1-0814f8289308 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.021262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:58.039464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.040075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.040994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.041347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.042082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:58.042389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.043869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.067675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 298/585] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:58.067675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.081377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c6821-1a69-4a6a-bc9c-b73f123c6bc8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.119895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.120539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.120819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.121109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.121830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=b92a2d92e27d4fe48b42f9e04be92372) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:58.122126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.123602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.129336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 288/586] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:35:57 2022] GET /identity/v3/role_assignments?user.id=b92a2d92e27d4fe48b42f9e04be92372&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:58.130106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.198532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f033a8-cb7f-4518-8af7-32a5e6360acb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.240194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.240837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.241369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.241635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.242288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:58.242576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.244250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.253469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:58.258438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 299/587] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:58.260199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.278998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c8dd4-c84f-47f5-8915-30b9fed9319f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.304867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:58.333493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.334446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.334731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.335043 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.335760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:58.336058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.338145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.354768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 289/588] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:58.355796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.392165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea15ee-6bd1-4e14-9a98-cda4a9e2727f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.430228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.431031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.431480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.431929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.444249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.445723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:58.463055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:58.470670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:58.471578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:58.471578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:58.471578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:58.471578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 300/589] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:58.471578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.485458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2148dba-6707-4ffe-bff9-3bf09a2aecba None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.530884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.532590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.533609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.534134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.544332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.547092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:58.563791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:58.564968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:35:58.566117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:58.567287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 290/590] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:58.572644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.576079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:58.627549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:58.644556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c0268-3fa0-459e-8b90-6dc0841fea75 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.693695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.694721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.695198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.695565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.701771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dc24d-33c4-406d-94f9-020968d26588 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.706433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.708030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:35:58.724397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:58.725593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:35:58.726759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:58.728267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 301/591] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:58.729134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.753926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.754689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.755099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.755472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.756600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:58.756992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.758860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.775022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 291/592] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:58.776224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.845179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:58.871192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1b40f-2d95-45b6-a6c8-6d1b430dd56b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.894475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b143b87-2ea0-42f2-8941-4986123b949d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:58.910085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.911054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.911476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.912012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.912926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:58.913388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.915895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.935983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:58.937261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:58.937634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:58.938002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:58.938827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:58.939229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 302/593] 10.222.0.94 () {56 vars in 1137 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:58.939343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:58.940357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:58.941040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:58.952104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:58.970952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 292/594] 10.222.0.94 () {56 vars in 1139 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:58.970952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.081441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0141b-1c61-467a-8791-83d886e5afa2 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:59.108455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8bfce-0523-4da8-ba58-51417fa6ed00 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:59.129468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:59.130532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.130912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.131331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.141741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:59.142680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:59.144045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:59.157135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:59.158000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.158425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.158858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:59.160064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:59.161090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:59.162219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:59.162219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:59.162219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:59.162219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:59.162219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:59.162219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:59.162219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:59.162219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:59.167001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 303/595] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:59.168049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.179138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:59.181599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:59.203926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:59.205099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:59.206187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:59.206187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:59.206187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:59.208348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 293/596] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:59.209537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.257717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:59.299331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2593dc-1844-4f6a-bb88-1dbc9aec647c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:59.335467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d664b4f-4949-41e9-b229-ba1d1958144e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:59.339313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:59.340139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.340581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.340994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.352487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:59.355119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:35:59.372026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:59.373126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:35:59.374052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:35:59.374052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:35:59.374052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:35:59.374823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:59.375538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.375891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.376231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.381568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:59.381568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:59.381568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:59.382129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 304/597] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:59.383347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.397696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 294/598] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:59.398854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.444835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:59.504927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7888fde-05eb-4470-9168-ab8d28fc68bb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:59.521513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323bf98-0026-4f08-b58b-c4ec8121d068 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:59.526284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:59.550278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:59.551276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.551738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.552188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.553152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:59.553783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:59.555687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:59.566868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8833c5-84af-4242-b805-97677784414c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:59.567678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8833c5-84af-4242-b805-97677784414c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.568023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8833c5-84af-4242-b805-97677784414c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.568361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8833c5-84af-4242-b805-97677784414c None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.569084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8833c5-84af-4242-b805-97677784414c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:35:59.569482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8833c5-84af-4242-b805-97677784414c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:59.575779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8833c5-84af-4242-b805-97677784414c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:59.575985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 305/599] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:59.575985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.587919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 295/600] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:35:59.587919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94741-7ebc-4e60-b823-015fbe76c0df None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.587919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94741-7ebc-4e60-b823-015fbe76c0df None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.587919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94741-7ebc-4e60-b823-015fbe76c0df None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.587919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 296/601] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:35:59.588732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a94741-7ebc-4e60-b823-015fbe76c0df None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.690582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abd448-2d6f-4858-88a1-820a1ea5f9a6 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:59.695319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:35:59.705481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a94741-7ebc-4e60-b823-015fbe76c0df None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:35:59.728723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9c29b-cd04-4c10-8182-f226d9a502fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:59.729399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c29b-cd04-4c10-8182-f226d9a502fc None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.729674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c29b-cd04-4c10-8182-f226d9a502fc None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.729944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c29b-cd04-4c10-8182-f226d9a502fc None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/500d2f2bd83d499aa2814c6d7cc71795/roles/b8b6b507d5284b84a25895b280b89cbd` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.744636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9212c34-b78f-4998-9e27-6795956158c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:35:59.745736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9212c34-b78f-4998-9e27-6795956158c8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.746018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9212c34-b78f-4998-9e27-6795956158c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.746136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c29b-cd04-4c10-8182-f226d9a502fc None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=500d2f2bd83d499aa2814c6d7cc71795, role_id=b8b6b507d5284b84a25895b280b89cbd)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:59.746289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9212c34-b78f-4998-9e27-6795956158c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.747739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c29b-cd04-4c10-8182-f226d9a502fc None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:59.753825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9212c34-b78f-4998-9e27-6795956158c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:35:59.755294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9212c34-b78f-4998-9e27-6795956158c8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e9212c34-b78f-4998-9e27-6795956158c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:35:59.766880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:35:59.772740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 297/602] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e6c0-fdb2-4529-9809-aa141e2d6d6f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:35:59.773851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e6c0-fdb2-4529-9809-aa141e2d6d6f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:35:59.774570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e6c0-fdb2-4529-9809-aa141e2d6d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:35:59.788940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ac27e6c0-fdb2-4529-9809-aa141e2d6d6f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:35:59.790716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ac27e6c0-fdb2-4529-9809-aa141e2d6d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:35:59.854984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 306/603] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:35:59 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/500d2f2bd83d499aa2814c6d7cc71795/roles/b8b6b507d5284b84a25895b280b89cbd => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:35:59.856243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9c29b-cd04-4c10-8182-f226d9a502fc None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.870035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 298/604] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:35:59.871310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27e6c0-fdb2-4529-9809-aa141e2d6d6f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:35:59.922832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:00.045985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:00.063655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9c29b-cd04-4c10-8182-f226d9a502fc None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:00.078180 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27e6c0-fdb2-4529-9809-aa141e2d6d6f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:00.108171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:00.110866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.111442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.111856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.121743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:00.123575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:00.125295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:00.126321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.126690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.127510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.128310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:00.128712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:00.130223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:00.139476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:00.140653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:00.141754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:00.141754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:00.141754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:00.141754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:00.141754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:00.141754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:00.141754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:00.141754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:00.143952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 307/605] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:00.146996 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 299/606] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:00.150060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:00.152838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:00.251267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:00.264637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4224520-5de9-4a29-97fb-ce146738077a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:00.274422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999f35-568e-4f43-a567-12c7d1176232 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:00.300087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbc6b8-8f91-40d0-bd3e-2ebdc2c71b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:00.300710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbc6b8-8f91-40d0-bd3e-2ebdc2c71b37 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.301064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbc6b8-8f91-40d0-bd3e-2ebdc2c71b37 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.301384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbc6b8-8f91-40d0-bd3e-2ebdc2c71b37 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.302140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbc6b8-8f91-40d0-bd3e-2ebdc2c71b37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:00.302501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbc6b8-8f91-40d0-bd3e-2ebdc2c71b37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:00.304018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbc6b8-8f91-40d0-bd3e-2ebdc2c71b37 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:00.310458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7440b7e-b450-46a4-bf32-060ec419aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:00.311234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7440b7e-b450-46a4-bf32-060ec419aeee None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.311584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7440b7e-b450-46a4-bf32-060ec419aeee None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.311922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7440b7e-b450-46a4-bf32-060ec419aeee None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/58567e28a5f741eda021aecab43d6a63/roles/b8b6b507d5284b84a25895b280b89cbd` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.321299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 308/607] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:00.321299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa8dfa-93fc-437d-98df-d57d47d3b9d4 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.321299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa8dfa-93fc-437d-98df-d57d47d3b9d4 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.321299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa8dfa-93fc-437d-98df-d57d47d3b9d4 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.321299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 309/608] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:00.321299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa8dfa-93fc-437d-98df-d57d47d3b9d4 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:00.333402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7440b7e-b450-46a4-bf32-060ec419aeee None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=58567e28a5f741eda021aecab43d6a63, role_id=b8b6b507d5284b84a25895b280b89cbd)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:00.334842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7440b7e-b450-46a4-bf32-060ec419aeee None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:00.337603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:00.428390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa8dfa-93fc-437d-98df-d57d47d3b9d4 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:00.467418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afefe4a-2c2b-4f25-9427-e0a083132ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:00.468062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afefe4a-2c2b-4f25-9427-e0a083132ee1 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.468713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 300/609] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:36:00 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/58567e28a5f741eda021aecab43d6a63/roles/b8b6b507d5284b84a25895b280b89cbd => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:36:00.468849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afefe4a-2c2b-4f25-9427-e0a083132ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.468849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afefe4a-2c2b-4f25-9427-e0a083132ee1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.480023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7440b7e-b450-46a4-bf32-060ec419aeee None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:00.483035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afefe4a-2c2b-4f25-9427-e0a083132ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-2afefe4a-2c2b-4f25-9427-e0a083132ee1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:00.489478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:00.490515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:00.491654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 310/610] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:00.492865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648126c-fcc1-42e8-b984-6ef7fbf0bcc3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.492865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648126c-fcc1-42e8-b984-6ef7fbf0bcc3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.493099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648126c-fcc1-42e8-b984-6ef7fbf0bcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.518004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5648126c-fcc1-42e8-b984-6ef7fbf0bcc3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:00.519977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5648126c-fcc1-42e8-b984-6ef7fbf0bcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:00.522990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:00.626759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 311/611] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:00.628533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648126c-fcc1-42e8-b984-6ef7fbf0bcc3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:00.654663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7440b7e-b450-46a4-bf32-060ec419aeee None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:00.703932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d05538-34f9-4795-b25e-45f44188faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:00.704599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d05538-34f9-4795-b25e-45f44188faba None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.704871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d05538-34f9-4795-b25e-45f44188faba None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.705139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d05538-34f9-4795-b25e-45f44188faba None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/b7ad758153334ec28940156d09495ae6/roles/b8b6b507d5284b84a25895b280b89cbd` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.725927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d05538-34f9-4795-b25e-45f44188faba None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=b7ad758153334ec28940156d09495ae6, role_id=b8b6b507d5284b84a25895b280b89cbd)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:00.727439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d05538-34f9-4795-b25e-45f44188faba None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:00.766151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648126c-fcc1-42e8-b984-6ef7fbf0bcc3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:00.815094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 301/612] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:36:00 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/b7ad758153334ec28940156d09495ae6/roles/b8b6b507d5284b84a25895b280b89cbd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:36:00.817873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:00.818501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:00.818772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:00.819057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:00.819720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:00.820013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:00.822415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:00.828901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:00.839534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 312/613] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:00.842011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d05538-34f9-4795-b25e-45f44188faba None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:00.913845 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:00.937697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:01.018876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d05538-34f9-4795-b25e-45f44188faba None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:01.084573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:01.085236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:01.085511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:01.085778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:01.107288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:01.108805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:01.113604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1b505-1312-434b-a1e6-390121d74017 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:01.138447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:01.139522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:01.140383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:01.140383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:01.140383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:01.140383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 302/614] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:01.146569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:01.173052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eb067-b597-4f47-900f-15bb97e86729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:01.173798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eb067-b597-4f47-900f-15bb97e86729 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:01.174157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eb067-b597-4f47-900f-15bb97e86729 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:01.174481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eb067-b597-4f47-900f-15bb97e86729 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:01.198835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eb067-b597-4f47-900f-15bb97e86729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-fa4eb067-b597-4f47-900f-15bb97e86729 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:01.212851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:01.213849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:01.214697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:01.217147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 313/615] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:01.223692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eb067-b597-4f47-900f-15bb97e86729 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:01.320902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:01.334564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f331dd-e506-4531-abc9-4c4996179325 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:01.351289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:01.406419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:01.407075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:01.407358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:01.407633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:01.408287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:01.408585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:01.409972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:01.416159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eb067-b597-4f47-900f-15bb97e86729 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:01.423104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 303/616] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:01.430270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:01.461766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:01.462872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:01.463288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:01.463651 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:01.464420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:01.464814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:01.466281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:01.475310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 314/617] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:01.481599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:01.631058 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242fa63-517a-481b-a377-fd14ef0f9004 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:01.663402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:01.674204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:01.674891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:01.682838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:01.682838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:01.683073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4b355-4f72-4187-a9a3-34f1849a5898 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:01.686799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:01.688300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:01.703444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:01.704418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:01.706113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:01.706113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:01.706113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:01.706113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 315/618] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:01.710770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:01.757015 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:01.757733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:01.757918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:01.758182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] PATH_INFO: `/v3/users/glance` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:01.776159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:01.777640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:01.791621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:01.792721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:01.794277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 304/619] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:01.800009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:01.841418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:01.854412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fbf6a-e07e-4e02-ade3-a94c762bc545 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:01.898223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:01.898836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:01.899133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:01.899400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:01.900721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:01.901116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:01.902617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:01.930036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 316/620] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:01.938806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:01.981196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99be078-df27-4ab5-9db2-3a597b963fed None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:02.007306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:02.036954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:02.037575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.038023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.038329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.039099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:02.039404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:02.041761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:02.064341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 305/621] 10.222.0.94 () {56 vars in 1137 bytes} [Fri Aug 26 15:36:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:02.072104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:02.133745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a196d-4a02-4a2c-8f4a-42cfaf8ea877 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:02.172980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:02.173959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.174377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.174786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.183899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:02.185545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:02.198195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:02.199247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:02.200197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:02.202995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 317/622] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:02.208122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:02.221341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:02.252508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ac253-2fe8-43b7-849b-3f1f21326034 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:02.290388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa627-f8df-4dcd-bb6f-988d389706a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:02.291121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa627-f8df-4dcd-bb6f-988d389706a7 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.291508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa627-f8df-4dcd-bb6f-988d389706a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.291785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa627-f8df-4dcd-bb6f-988d389706a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.323333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:02.330886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d815b11-a6d9-4a90-97a6-1687b354ff27 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:02.367211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa627-f8df-4dcd-bb6f-988d389706a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:02.369514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfa627-f8df-4dcd-bb6f-988d389706a7 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:02.370379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf827879-4a94-490d-9669-9328885e9a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:02.371345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf827879-4a94-490d-9669-9328885e9a13 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.372117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf827879-4a94-490d-9669-9328885e9a13 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.372694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf827879-4a94-490d-9669-9328885e9a13 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.373621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf827879-4a94-490d-9669-9328885e9a13 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:02.374144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf827879-4a94-490d-9669-9328885e9a13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:02.375777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf827879-4a94-490d-9669-9328885e9a13 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:02.395630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 318/623] 10.222.0.94 () {56 vars in 1135 bytes} [Fri Aug 26 15:36:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:02.397392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a5204-6866-480b-b1e1-5b3f50a7e2b3 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.397898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a5204-6866-480b-b1e1-5b3f50a7e2b3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.398426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a5204-6866-480b-b1e1-5b3f50a7e2b3 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.400806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 319/624] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:02.401865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a5204-6866-480b-b1e1-5b3f50a7e2b3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:02.456793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:02.490339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 306/625] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:36:02.490339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250d0f4-a581-4057-a1bf-570d27aecb62 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.490339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250d0f4-a581-4057-a1bf-570d27aecb62 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.490339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250d0f4-a581-4057-a1bf-570d27aecb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.507434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3250d0f4-a581-4057-a1bf-570d27aecb62 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:02.508692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3250d0f4-a581-4057-a1bf-570d27aecb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:02.531017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a5204-6866-480b-b1e1-5b3f50a7e2b3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:02.572414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:02.573487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.574326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.574798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.587777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:02.589588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:02.595367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 307/626] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:02.596370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250d0f4-a581-4057-a1bf-570d27aecb62 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:02.606984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:02.607975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:02.608859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:02.608859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:02.608859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:02.611013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 320/627] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:02.620049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:02.622100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:02.736828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519140c3-b42c-4809-9191-c8b37ed19f82 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:02.773743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c91b1-359b-4329-8d3b-ff86f11c0a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:02.774441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c91b1-359b-4329-8d3b-ff86f11c0a3b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.775483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c91b1-359b-4329-8d3b-ff86f11c0a3b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.775759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c91b1-359b-4329-8d3b-ff86f11c0a3b None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.776624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c91b1-359b-4329-8d3b-ff86f11c0a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:02.776931 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c91b1-359b-4329-8d3b-ff86f11c0a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:02.778323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c91b1-359b-4329-8d3b-ff86f11c0a3b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:02.783412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250d0f4-a581-4057-a1bf-570d27aecb62 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:02.785992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:02.791446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 321/628] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:02.792737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a98d-1e5e-40f6-81e3-6a633018d8a1 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.793022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a98d-1e5e-40f6-81e3-6a633018d8a1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.793292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801a98d-1e5e-40f6-81e3-6a633018d8a1 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.800597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 322/629] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:02.801972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801a98d-1e5e-40f6-81e3-6a633018d8a1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:02.830426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c71b2-98c3-4653-853b-eec9723ccb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:02.831362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c71b2-98c3-4653-853b-eec9723ccb8b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.831642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c71b2-98c3-4653-853b-eec9723ccb8b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.831914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c71b2-98c3-4653-853b-eec9723ccb8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.846650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c71b2-98c3-4653-853b-eec9723ccb8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:02.848653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c71b2-98c3-4653-853b-eec9723ccb8b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-3e9c71b2-98c3-4653-853b-eec9723ccb8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:02.870513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:02.872349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:02.873420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:02.873420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:02.873420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:02.873603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:02.873689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 308/630] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:02.877049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e8bf1-5277-4298-9ca2-79155cc487b7 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.877049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e8bf1-5277-4298-9ca2-79155cc487b7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.877049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e8bf1-5277-4298-9ca2-79155cc487b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.901312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-fb3e8bf1-5277-4298-9ca2-79155cc487b7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:02.902530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-fb3e8bf1-5277-4298-9ca2-79155cc487b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:02.925806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801a98d-1e5e-40f6-81e3-6a633018d8a1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:02.972158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:02.972847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:02.973130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:02.973403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:02.980954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:02.987269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 309/631] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:02.987269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e8bf1-5277-4298-9ca2-79155cc487b7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:02.987485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:03.011014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:03.012166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:03.013113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:03.013113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:03.013113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:03.013113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:03.013113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:03.013113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:03.013113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:03.013113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:03.017399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 323/632] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:03.018649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.092822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:03.108542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e8bf1-5277-4298-9ca2-79155cc487b7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.145708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.146862 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.147252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.147760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.148416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:03.149124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:03.151326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:03.166959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 310/633] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:03.167266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23aca7-36e7-4b3a-8dde-516efdd3e44c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.167391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.194312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:03.206750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eed7fb-215c-4e84-9f5f-e80d5a2d7805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.207414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed7fb-215c-4e84-9f5f-e80d5a2d7805 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.207711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed7fb-215c-4e84-9f5f-e80d5a2d7805 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.207958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eed7fb-215c-4e84-9f5f-e80d5a2d7805 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.219707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eed7fb-215c-4e84-9f5f-e80d5a2d7805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-f7eed7fb-215c-4e84-9f5f-e80d5a2d7805 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:03.224821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:03.225739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:03.226574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:03.230775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 324/634] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:03.232071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f81e33-70d0-49e1-9cc1-51f50d2db041 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.232355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f81e33-70d0-49e1-9cc1-51f50d2db041 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.232630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f81e33-70d0-49e1-9cc1-51f50d2db041 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.233902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 325/635] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:03.238094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f81e33-70d0-49e1-9cc1-51f50d2db041 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.282883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1e810-a79d-4281-ba81-25e81368e24c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.318547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.319181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.319454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.319724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.320410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:03.320705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:03.322713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:03.334799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 311/636] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:03.335739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.344191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:03.349086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f81e33-70d0-49e1-9cc1-51f50d2db041 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.389962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f791992-5375-4405-970c-87416d7eb787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.390951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791992-5375-4405-970c-87416d7eb787 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.391406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791992-5375-4405-970c-87416d7eb787 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.391845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791992-5375-4405-970c-87416d7eb787 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.400256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f791992-5375-4405-970c-87416d7eb787 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:03.401750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f791992-5375-4405-970c-87416d7eb787 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-2f791992-5375-4405-970c-87416d7eb787 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:03.413556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:03.416066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 326/637] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197c6d-430c-4ab8-8405-51babc597967 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.418066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197c6d-430c-4ab8-8405-51babc597967 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.419447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197c6d-430c-4ab8-8405-51babc597967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.435529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8a197c6d-430c-4ab8-8405-51babc597967 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:03.440145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8a197c6d-430c-4ab8-8405-51babc597967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:03.451571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eda6fc-2e50-485a-9cd9-cc58f4624572 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.479440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:03.488888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.489571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.489911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.490244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.491270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:03.491666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:03.493092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:03.501655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 312/638] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:03.502576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.508027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 327/639] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:03.509387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197c6d-430c-4ab8-8405-51babc597967 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.583108 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:03.618046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197c6d-430c-4ab8-8405-51babc597967 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.618357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2815e10-1f26-4610-8583-87ce0cfc77b4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.653314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe6a95-e764-4ff7-b0ed-037404a663cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.653636 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6ac63-42c1-44f8-a720-d25744bd41ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.653954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe6a95-e764-4ff7-b0ed-037404a663cf None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.654246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe6a95-e764-4ff7-b0ed-037404a663cf None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.654516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe6a95-e764-4ff7-b0ed-037404a663cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.655124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6ac63-42c1-44f8-a720-d25744bd41ba None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.655585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6ac63-42c1-44f8-a720-d25744bd41ba None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.656037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6ac63-42c1-44f8-a720-d25744bd41ba None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/b92a2d92e27d4fe48b42f9e04be92372/roles/b8b6b507d5284b84a25895b280b89cbd` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.664824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe6a95-e764-4ff7-b0ed-037404a663cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-2ebe6a95-e764-4ff7-b0ed-037404a663cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:03.670475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:03.671549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:03.672387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 313/640] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:03.673101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe6a95-e764-4ff7-b0ed-037404a663cf None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.675017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6ac63-42c1-44f8-a720-d25744bd41ba None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=b92a2d92e27d4fe48b42f9e04be92372, role_id=b8b6b507d5284b84a25895b280b89cbd)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:03.676871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6ac63-42c1-44f8-a720-d25744bd41ba None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:03.773747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe6a95-e764-4ff7-b0ed-037404a663cf None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.808915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.809586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.809784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.810055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:03.810737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:03.811077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:03.812513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:03.824831 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 314/641] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:03.825610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.836663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 328/642] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:36:03 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/b92a2d92e27d4fe48b42f9e04be92372/roles/b8b6b507d5284b84a25895b280b89cbd => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:36:03.838425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6ac63-42c1-44f8-a720-d25744bd41ba None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:03.855737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:03.867965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:03.941080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4ddfe-5dec-4b69-bad7-1339729c59b0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.952184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6ac63-42c1-44f8-a720-d25744bd41ba None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:03.987217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:03.988134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:03.988409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:03.988671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.000314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.000314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.000314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.000999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.001090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:04.001356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.002115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.002705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.003620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.011365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 329/643] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:04.015719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:04.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:04.020540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:04.021561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:04.021561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:04.021561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:04.021561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 315/644] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:04.021561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.068778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:04.117471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:04.132646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330588d-cfac-4542-9172-cc892655a69d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.134067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f91e3-c8dd-497f-9b07-b468a13dafc0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.170951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.171659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.171854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5b946-02e4-421f-82e6-f6beac6fffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.172066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.172405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.172514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5b946-02e4-421f-82e6-f6beac6fffff None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.172749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5b946-02e4-421f-82e6-f6beac6fffff None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.173022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5b946-02e4-421f-82e6-f6beac6fffff None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.173368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=23ddf8f600454da7ae9b715cd4cbd186) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:04.173768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.176589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.191157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 316/645] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/role_assignments?user.id=23ddf8f600454da7ae9b715cd4cbd186&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:04.191380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5b946-02e4-421f-82e6-f6beac6fffff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:04.191489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-46c5b946-02e4-421f-82e6-f6beac6fffff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:04.191584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:04.192618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:04.193781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:04.195145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 330/646] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:04.195953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5b946-02e4-421f-82e6-f6beac6fffff None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.329272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5b946-02e4-421f-82e6-f6beac6fffff None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.344060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6785e2-fe2c-4a6f-a431-c6e19bad2697 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.347329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:04.385527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.386141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.386340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.386452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.386666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.387006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.387293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.387392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:04.387561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.387665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.388978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.401312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.402426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 331/647] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:04.402802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.403450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.414556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:04.423156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:04.424144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:04.425084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:04.425084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:04.425084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:04.425084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 317/648] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:04.427217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.542252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9731a-1ac5-4132-90a3-9beb410fbdcb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.570347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c168a4e-c1a5-4034-b04c-c4069be00a32 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.579947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.580506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.580782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.581051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.581717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:04.582018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.583423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.592984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 332/649] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:04.593927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.611249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.611887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.612475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.612752 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.621852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.623302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:04.636014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:04.637006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:04.637951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 318/650] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:04.638543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.665653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:04.690637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:04.732686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd7245-0253-419b-8240-e119a59a815a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.773276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.774220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.774752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.775219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.776064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:04.776556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.778024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f49fac-161a-4141-a306-62690253c746 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.778315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.792239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 333/651] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:04.793495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.818092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.818804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.819037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.819362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.833322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.834763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:04.853145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:04.854299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:04.855210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:04.855210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:04.855210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:04.858973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 319/652] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:04.859867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:04.916044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0564cd-68fe-4e38-a2f5-73023db6b794 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:04.945019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:04.957887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:04.958946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:04.960124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:04.962559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:04.962559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:04.962559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:04.966968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:04.966968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:04.983259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 334/653] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:04.984228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.003539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cce15-45cf-4807-ac46-02bbd6beae6f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.042134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.042905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.043207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.043599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.051823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.053537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:05.066114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:05.067092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:05.068012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 320/654] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:05.068596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.098273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57c387-b720-4ce6-b1c2-2d49b9be5817 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.136490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.137104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.137381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.137649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.138292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:05.138588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.140104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.153389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 335/655] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:05.154250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.183241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6855d3-1d65-4d7c-a03f-7e297c43b154 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.205281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:05.212527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:05.220636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1087-cbaf-42c0-9539-7924aff03c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.222323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1087-cbaf-42c0-9539-7924aff03c48 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.222323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1087-cbaf-42c0-9539-7924aff03c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.222323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1087-cbaf-42c0-9539-7924aff03c48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.266106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487e6ec-e79d-48fb-8d77-fc8c8f21a586 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.297385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e1087-cbaf-42c0-9539-7924aff03c48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.299073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e1087-cbaf-42c0-9539-7924aff03c48 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.304282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.304963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.305242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.305510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.306794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:05.307108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.308769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.328814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 336/656] 10.222.0.94 () {56 vars in 1137 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:05.329668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.409945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 321/657] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:36:05.410907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1087-cbaf-42c0-9539-7924aff03c48 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.441876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:05.448566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae6f3f-e65a-4865-a1d8-9f6eab587b11 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.451459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:05.487345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.488008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.488285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.488557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.497697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.499365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:05.513265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:05.514460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 15:36:05.515698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:05.519218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 337/658] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:05.520156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.528919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1087-cbaf-42c0-9539-7924aff03c48 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.567580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d651-f04c-4357-a7a7-f9b84ca5fe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.568611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d651-f04c-4357-a7a7-f9b84ca5fe08 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.569078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d651-f04c-4357-a7a7-f9b84ca5fe08 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.569491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d651-f04c-4357-a7a7-f9b84ca5fe08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.630354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310928-2297-4698-b7e2-f210f3762bde None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.645508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41d651-f04c-4357-a7a7-f9b84ca5fe08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.647200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41d651-f04c-4357-a7a7-f9b84ca5fe08 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.668687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.669187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.669544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.669912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.684283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.685894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:05.701960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:05.703201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:05.704203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:05.704203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:05.704203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:05.704406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:05.709921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 338/659] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:05.710968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.731641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:05.764571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 322/660] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:36:05.765866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d651-f04c-4357-a7a7-f9b84ca5fe08 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.825286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f240cf7-60a0-49fd-ba58-84d1fbc21906 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.870208 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.870827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.871134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.871406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.872143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:05.872441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.873885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.889323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d651-f04c-4357-a7a7-f9b84ca5fe08 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:05.894812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 339/661] 10.222.0.94 () {56 vars in 1139 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:05.895685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.925384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:05.935301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc705a-d635-4ada-bb2e-642cdb5f3fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:05.936784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc705a-d635-4ada-bb2e-642cdb5f3fcf None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.938560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc705a-d635-4ada-bb2e-642cdb5f3fcf None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.938560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc705a-d635-4ada-bb2e-642cdb5f3fcf None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.938560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc705a-d635-4ada-bb2e-642cdb5f3fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:05.938560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc705a-d635-4ada-bb2e-642cdb5f3fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:05.939603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc705a-d635-4ada-bb2e-642cdb5f3fcf None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:05.953757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 323/662] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:05.955033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7281f-bb8c-481b-8e6d-55549284c4eb None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:05.955316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7281f-bb8c-481b-8e6d-55549284c4eb None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:05.955587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7281f-bb8c-481b-8e6d-55549284c4eb None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:05.957042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 324/663] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:05.957691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7281f-bb8c-481b-8e6d-55549284c4eb None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:05.987146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:06.014542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e06afb-8aff-465a-a264-2b698b30e60d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:06.052549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:06.053189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.053467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.053736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.067118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:06.068578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:06.073397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7281f-bb8c-481b-8e6d-55549284c4eb None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:06.086837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:06.087900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:06.088791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:06.088791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:06.088791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:06.092812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 340/664] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:06.093594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:06.114012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7497c-5096-428f-af60-1147d529bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:06.114619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7497c-5096-428f-af60-1147d529bc12 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.114885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7497c-5096-428f-af60-1147d529bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.115182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7497c-5096-428f-af60-1147d529bc12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.177734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:06.192576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7497c-5096-428f-af60-1147d529bc12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:06.194135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7497c-5096-428f-af60-1147d529bc12 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:06.203298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207dc4-9b8f-4bc8-af59-83c0176cbcba None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:06.231940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:06.241132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744dd35-c078-45f7-9bbc-85bd30040df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:06.247100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744dd35-c078-45f7-9bbc-85bd30040df4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.247100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744dd35-c078-45f7-9bbc-85bd30040df4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.247100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744dd35-c078-45f7-9bbc-85bd30040df4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.249882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744dd35-c078-45f7-9bbc-85bd30040df4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:06.251800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744dd35-c078-45f7-9bbc-85bd30040df4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-3744dd35-c078-45f7-9bbc-85bd30040df4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:06.264554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:06.270548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 341/665] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5efb36-6105-493f-936c-0c58e41c1240 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.271737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5efb36-6105-493f-936c-0c58e41c1240 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.272495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5efb36-6105-493f-936c-0c58e41c1240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.288187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-df5efb36-6105-493f-936c-0c58e41c1240 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:06.289381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-df5efb36-6105-493f-936c-0c58e41c1240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:06.319573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 325/666] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:36:06.319573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7497c-5096-428f-af60-1147d529bc12 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:06.375111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 342/667] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:06.375111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5efb36-6105-493f-936c-0c58e41c1240 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:06.429414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:06.446327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7497c-5096-428f-af60-1147d529bc12 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:06.494805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:06.495750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.496031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.496302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.497213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:06.497521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:06.499643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:06.501525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5efb36-6105-493f-936c-0c58e41c1240 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:06.515077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 326/668] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:06.515935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:06.520914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:06.543299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd93949-072d-4406-9136-f7cac4eb2ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:06.544046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd93949-072d-4406-9136-f7cac4eb2ee8 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.544327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd93949-072d-4406-9136-f7cac4eb2ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.544597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd93949-072d-4406-9136-f7cac4eb2ee8 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.545504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd93949-072d-4406-9136-f7cac4eb2ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:06.545801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd93949-072d-4406-9136-f7cac4eb2ee8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:06.547882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd93949-072d-4406-9136-f7cac4eb2ee8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:06.574004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 343/669] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:06.575520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195cfd1-860a-45f4-9a21-5def77e1ed37 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.575733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195cfd1-860a-45f4-9a21-5def77e1ed37 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.575952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195cfd1-860a-45f4-9a21-5def77e1ed37 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.577350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 344/670] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:06.577900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195cfd1-860a-45f4-9a21-5def77e1ed37 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:06.641861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b3c31-855b-46f8-98d6-e59a79cd1e59 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:06.679077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:06.679778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.679982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.680248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.691455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:06.692917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:06.693030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195cfd1-860a-45f4-9a21-5def77e1ed37 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:06.703865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:06.710305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:06.711473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:06.712954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:06.712954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:06.712954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:06.712954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 327/671] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:06.712954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:06.732065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1593b1-8509-42ab-85ca-2c645f0a1833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:06.732701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1593b1-8509-42ab-85ca-2c645f0a1833 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.733006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1593b1-8509-42ab-85ca-2c645f0a1833 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.733238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1593b1-8509-42ab-85ca-2c645f0a1833 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.740540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1593b1-8509-42ab-85ca-2c645f0a1833 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:06.742231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1593b1-8509-42ab-85ca-2c645f0a1833 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-cb1593b1-8509-42ab-85ca-2c645f0a1833 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:06.754021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:06.755098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:06.756578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 345/672] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:06.758404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b5ec9-6048-4b48-942f-7fb7ecc34efb None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.758779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b5ec9-6048-4b48-942f-7fb7ecc34efb None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.759098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b5ec9-6048-4b48-942f-7fb7ecc34efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.777116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3a8b5ec9-6048-4b48-942f-7fb7ecc34efb None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:06.778345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3a8b5ec9-6048-4b48-942f-7fb7ecc34efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:06.808923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:06.817714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0eac-2ab4-4ac7-a8ce-945e55f0866a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:06.848170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 346/673] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:06.849031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b5ec9-6048-4b48-942f-7fb7ecc34efb None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:06.861013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:06.861640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:06.861934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:06.862188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:06.862906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=500d2f2bd83d499aa2814c6d7cc71795) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:06.863239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:06.864900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:06.871938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 328/674] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:06 2022] GET /identity/v3/role_assignments?user.id=500d2f2bd83d499aa2814c6d7cc71795&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:06.872740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:06.954053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:06.982733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b5ec9-6048-4b48-942f-7fb7ecc34efb None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.004792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123012a-6e81-4465-a26d-3a49e0404fc1 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.036708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654d490-42bf-4b82-b6a4-d3d51ccbe13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.037350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654d490-42bf-4b82-b6a4-d3d51ccbe13b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.037625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654d490-42bf-4b82-b6a4-d3d51ccbe13b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.037879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654d490-42bf-4b82-b6a4-d3d51ccbe13b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.055002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654d490-42bf-4b82-b6a4-d3d51ccbe13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:07.055220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.055220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.056356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.057384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.057384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:07.058680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-7654d490-42bf-4b82-b6a4-d3d51ccbe13b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:07.059451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:07.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:07.061595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:07.062380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.064528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 347/675] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:07.067880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654d490-42bf-4b82-b6a4-d3d51ccbe13b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.080404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 329/676] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:07.081946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.107719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:07.198557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654d490-42bf-4b82-b6a4-d3d51ccbe13b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.201494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23def5-0850-4c13-9967-c3d668e419bf None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.227573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:07.236186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.238769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.238769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.238769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.238769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:07.238769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.239762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.239959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.240663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.241288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.241727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.242688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] RBAC: Adding query filter params (name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:07.243062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.244528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.251928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 348/677] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:07.252763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.260517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 330/678] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:07.261622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.351336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:07.368759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d716ba9-37fc-4dec-b0b9-8b39fce895f3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.376687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6585d8-8f75-4520-a36c-a95ed18b6a39 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.405376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.406181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.406456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.406718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.414298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.415528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.416202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.417104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.417355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.417453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.430125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:07.430309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:07.437248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:07.438371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:07.438371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:07.438371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:07.438371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:07.438371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:07.438371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:07.438371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:07.438371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:07.438896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.438998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 349/679] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:07.438998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:07.452076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:07.453055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:07.454040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:07.454040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:07.454040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:07.454040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 331/680] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:07.454040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.504349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:07.559160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5d6cf-7eaf-4c20-afe4-055b3714c460 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.569800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92a2c8-a735-4c71-bf97-80ac21adead3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.610580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.611792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.612073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.612346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.613081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=58567e28a5f741eda021aecab43d6a63) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:07.613430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.614889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.621059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.622151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.622263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.622536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.631497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 350/681] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/role_assignments?user.id=58567e28a5f741eda021aecab43d6a63&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:07.632366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.637956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.639873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:07.659751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:07.660746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:07.661646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:07.661646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:07.661646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:07.664085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 332/682] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:07.665310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.668246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:07.782853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:07.795882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3993c0-f709-45eb-bcfc-382963dcf1be None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.797279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68436-c5ac-4558-b8c0-0efce285127c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:07.847543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.848798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.849093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.849332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.850025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:07.850324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.851961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.856670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:07.857410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:07.857788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:07.858211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:07.859025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:07.859427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:07.861437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:07.867180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 351/683] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:07.867327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.876658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 333/684] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:07.877657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:07.954778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:07.988888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b352c80-8925-40ce-9226-5b5b4c46f3df None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:08.001401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0740-003f-4d5f-bd8a-f3cd0d2fc1ca None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:08.027701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:08.028562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:08.028985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:08.029418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:08.030205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:08.030647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:08.032178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:08.038034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:08.038622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:08.038898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:08.039192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:08.040469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=b7ad758153334ec28940156d09495ae6) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:08.040819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:08.042281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:08.054953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 334/686] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/role_assignments?user.id=b7ad758153334ec28940156d09495ae6&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:08.054953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:08.055266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 352/686] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:08.064273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:08.097714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:08.218176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d65c19-9206-4cdb-89b0-469c87404654 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:08.277921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:08.277921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:08.278656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:08.278656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] PATH_INFO: `/v3/users/swift` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:08.283256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:08.286756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5845c99-bafb-48da-befa-a052dc60488c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:08.293741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:08.295393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:08.312837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:08.314122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:08.315360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 353/687] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:08.319148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:08.343637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c283c-880c-46c0-be70-d2ec79aea074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:08.344336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c283c-880c-46c0-be70-d2ec79aea074 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:08.344528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c283c-880c-46c0-be70-d2ec79aea074 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:08.344853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c283c-880c-46c0-be70-d2ec79aea074 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:08.345528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c283c-880c-46c0-be70-d2ec79aea074 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:08.347069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c283c-880c-46c0-be70-d2ec79aea074 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:08.437103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f25c94-5e08-4768-ac95-09c07e25eaeb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:08.474189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:08.475186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:08.475610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:08.476025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:08.476929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:08.477281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 335/688] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:36:08 2022] POST /identity/v3/users => generated 254 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:08.478248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:08.479981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:08.501032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 354/689] 10.222.0.94 () {56 vars in 1135 bytes} [Fri Aug 26 15:36:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:08.506765 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c283c-880c-46c0-be70-d2ec79aea074 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:08.652924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:08.686104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c283c-880c-46c0-be70-d2ec79aea074 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:08.723237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395378eb-6fc7-417e-b057-ae6ba2ac6274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:08.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395378eb-6fc7-417e-b057-ae6ba2ac6274 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:08.724127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395378eb-6fc7-417e-b057-ae6ba2ac6274 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:08.724389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395378eb-6fc7-417e-b057-ae6ba2ac6274 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:08.803409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395378eb-6fc7-417e-b057-ae6ba2ac6274 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:08.804933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395378eb-6fc7-417e-b057-ae6ba2ac6274 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:08.879571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:08.914318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 336/690] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:36:08.920725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:08.985672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:09.032553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38636f-49a3-4392-a7c2-4a360501f380 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:09.033066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38636f-49a3-4392-a7c2-4a360501f380 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:09.033141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38636f-49a3-4392-a7c2-4a360501f380 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:09.035734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 337/691] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:09.051829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dcb48-b307-4bb9-bd7f-c3d92c8d1c70 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:09.052146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dcb48-b307-4bb9-bd7f-c3d92c8d1c70 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:09.052375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dcb48-b307-4bb9-bd7f-c3d92c8d1c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:09.055939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2c694-ce1a-44c0-b2fe-ef3bd26658ee None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:09.091672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:09.092313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:09.092585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:09.092690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8e7dcb48-b307-4bb9-bd7f-c3d92c8d1c70 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:09.092899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:09.094161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8e7dcb48-b307-4bb9-bd7f-c3d92c8d1c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:09.104745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:09.106221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:09.122181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:09.123198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:09.124067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:09.124067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:09.124067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:09.124067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 355/692] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:09.128945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:09.193906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 338/693] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:36:09.238777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:09.242327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ce1a7-b40d-452c-80b6-af2a45ee05c7 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:09.278673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:09.279509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:09.279669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:09.279879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:09.280535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:09.280832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:09.282189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:09.295477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 356/694] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:09.300923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dcb48-b307-4bb9-bd7f-c3d92c8d1c70 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:09.403243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:09.409855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dcb48-b307-4bb9-bd7f-c3d92c8d1c70 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:09.460276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:09.460847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:09.461119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:09.461389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:09.476276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:09.477970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:09.491672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:09.492612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:09.493522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 339/695] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:09.498661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:09.512085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:09.664527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85fcec-0c86-4840-aeb8-b4cb5e6ef04b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:09.668307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be562-0ee8-478f-b4cf-a4ec422ac2c9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:09.703318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c1870-2b09-4851-a109-845b02b7b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:09.703951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1870-2b09-4851-a109-845b02b7b13a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:09.704224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1870-2b09-4851-a109-845b02b7b13a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:09.704490 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c1870-2b09-4851-a109-845b02b7b13a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:09.723930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c1870-2b09-4851-a109-845b02b7b13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-f74c1870-2b09-4851-a109-845b02b7b13a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:09.729408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:09.730333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:09.731167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:09.736483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 340/696] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:09.738453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:09.739098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:09.739372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:09.739639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:09.740397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:09.740953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:09.742374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:09.745608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c1870-2b09-4851-a109-845b02b7b13a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:09.770524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 357/697] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:09.777446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:09.786090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:09.802780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:09.898665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d42232-94f9-4350-871d-11d5fefa66fe None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:09.948479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9334b8-524d-48cc-98fa-0977899ec66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:09.953335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9334b8-524d-48cc-98fa-0977899ec66e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:09.953335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9334b8-524d-48cc-98fa-0977899ec66e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:09.953335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9334b8-524d-48cc-98fa-0977899ec66e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:09.953335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9334b8-524d-48cc-98fa-0977899ec66e None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=b92a2d92e27d4fe48b42f9e04be92372) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:09.953335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9334b8-524d-48cc-98fa-0977899ec66e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:09.953335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9334b8-524d-48cc-98fa-0977899ec66e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:09.981804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 358/698] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:09 2022] GET /identity/v3/role_assignments?user.id=b92a2d92e27d4fe48b42f9e04be92372&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:10.019743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c1870-2b09-4851-a109-845b02b7b13a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:10.023748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a742-afe2-4609-a9cf-9ba06caef0ad None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:10.026519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a742-afe2-4609-a9cf-9ba06caef0ad None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:10.026519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a742-afe2-4609-a9cf-9ba06caef0ad None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:10.026519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 359/699] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:10.032920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24138d72-ccff-44ee-b6bf-c90d4800d146 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:10.033340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24138d72-ccff-44ee-b6bf-c90d4800d146 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:10.035931 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24138d72-ccff-44ee-b6bf-c90d4800d146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:10.085422 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-24138d72-ccff-44ee-b6bf-c90d4800d146 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:10.086831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-24138d72-ccff-44ee-b6bf-c90d4800d146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:10.182933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:10.183652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:10.184169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:10.184519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:10.185678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:10.186295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:10.188959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:10.202260 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 341/700] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:10.212679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:10.221820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 360/701] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 15:36:10.367322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:10.375371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f086d-897f-4006-b90e-d4458ecaa610 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:10.424023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:10.424807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:10.425112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:10.425379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:10.439070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:10.442193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:10.465031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:10.466012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:10.466865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:10.466865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:10.466865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:10.472560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 342/702] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:10.474179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24138d72-ccff-44ee-b6bf-c90d4800d146 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:10.541251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:10.558964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:10.737122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24138d72-ccff-44ee-b6bf-c90d4800d146 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:10.772020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b2153-7e79-439d-9ba8-9fdb6e7dc817 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:10.783275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c012fc8-35c0-4351-984d-a8654e67b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:10.784628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c012fc8-35c0-4351-984d-a8654e67b004 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:10.786013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c012fc8-35c0-4351-984d-a8654e67b004 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:10.787199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c012fc8-35c0-4351-984d-a8654e67b004 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:10.787885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c012fc8-35c0-4351-984d-a8654e67b004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:10.788187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c012fc8-35c0-4351-984d-a8654e67b004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:10.789601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c012fc8-35c0-4351-984d-a8654e67b004 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:10.818600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0e1cc-4d60-4d22-b22e-e794d880c131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:10.819560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e1cc-4d60-4d22-b22e-e794d880c131 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:10.820057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e1cc-4d60-4d22-b22e-e794d880c131 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:10.820165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 361/703] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:10.820625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0e1cc-4d60-4d22-b22e-e794d880c131 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:10.821796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0e1cc-4d60-4d22-b22e-e794d880c131 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:10.822320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246af513-cf30-4574-855b-39ed1e358daa None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:10.822793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246af513-cf30-4574-855b-39ed1e358daa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:10.823313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246af513-cf30-4574-855b-39ed1e358daa None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:10.824156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0e1cc-4d60-4d22-b22e-e794d880c131 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:10.829737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 362/704] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:10.829737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246af513-cf30-4574-855b-39ed1e358daa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:10.831324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-44c0e1cc-4d60-4d22-b22e-e794d880c131 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:10.845891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:10.849793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:10.850628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 343/705] 10.222.0.94 () {56 vars in 1129 bytes} [Fri Aug 26 15:36:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:10.852508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c73c7-c83e-403e-8ab3-7e10b98a0ddb None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:10.853042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c73c7-c83e-403e-8ab3-7e10b98a0ddb None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:10.853513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c73c7-c83e-403e-8ab3-7e10b98a0ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:10.875821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5a8c73c7-c83e-403e-8ab3-7e10b98a0ddb None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:10.878191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5a8c73c7-c83e-403e-8ab3-7e10b98a0ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:10.965647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:10.982567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 344/706] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:10.982567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c73c7-c83e-403e-8ab3-7e10b98a0ddb None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:11.017927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246af513-cf30-4574-855b-39ed1e358daa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:11.101209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:11.102314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.102875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.103443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.114108 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c73c7-c83e-403e-8ab3-7e10b98a0ddb None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:11.125517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:11.127412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:11.145579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:11.148602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:11.150234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:11.151313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:11.156213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 363/707] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:36:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:11.158010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:11.158793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.159258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.159670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.160440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:11.160894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:11.161506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:11.162663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:11.168858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 345/708] 10.222.0.94 () {56 vars in 1124 bytes} [Fri Aug 26 15:36:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:11.175343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:11.313076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbac18a-4445-4492-b89d-dae280e94ba8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:11.317919 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dd7c0-9466-4298-97e6-877e586e1994 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:11.353947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4610a0d-1ba7-4df6-81a5-058e8969cf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:11.354517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4610a0d-1ba7-4df6-81a5-058e8969cf8a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.354626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b90b67-0838-4c11-bd12-3be359fcc869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:11.354844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4610a0d-1ba7-4df6-81a5-058e8969cf8a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.355079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4610a0d-1ba7-4df6-81a5-058e8969cf8a None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.355246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b90b67-0838-4c11-bd12-3be359fcc869 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.355484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b90b67-0838-4c11-bd12-3be359fcc869 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.355712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4610a0d-1ba7-4df6-81a5-058e8969cf8a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:11.355806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b90b67-0838-4c11-bd12-3be359fcc869 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.356003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4610a0d-1ba7-4df6-81a5-058e8969cf8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:11.356462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b90b67-0838-4c11-bd12-3be359fcc869 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:11.356882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b90b67-0838-4c11-bd12-3be359fcc869 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:11.357355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4610a0d-1ba7-4df6-81a5-058e8969cf8a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:11.358202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b90b67-0838-4c11-bd12-3be359fcc869 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:11.360325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:11.368900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 346/709] 10.222.0.94 () {56 vars in 1124 bytes} [Fri Aug 26 15:36:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:11.368900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c85b-873b-4e42-bf2c-b24d560ed5b7 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.368900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c85b-873b-4e42-bf2c-b24d560ed5b7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.368900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c85b-873b-4e42-bf2c-b24d560ed5b7 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.368900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 347/710] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:11.368900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298c85b-873b-4e42-bf2c-b24d560ed5b7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:11.386004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 364/711] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:36:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:11.387614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177373e-41b9-4892-9a9d-dad108a12d51 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.387895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177373e-41b9-4892-9a9d-dad108a12d51 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.388161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177373e-41b9-4892-9a9d-dad108a12d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.408535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4177373e-41b9-4892-9a9d-dad108a12d51 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:11.409864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4177373e-41b9-4892-9a9d-dad108a12d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:11.441630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:11.495353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298c85b-873b-4e42-bf2c-b24d560ed5b7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:11.499077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 365/712] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:11.501384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177373e-41b9-4892-9a9d-dad108a12d51 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:11.565778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:11.566354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.566628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.566896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] PATH_INFO: `/v3/services/network` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.567577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:11.576730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:11.591071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:11.592114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:11.592934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:11.595582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 348/713] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:11 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:11.600716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:11.708837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177373e-41b9-4892-9a9d-dad108a12d51 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:11.756106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b13b-56c7-4795-b3f8-0bac9d5f16eb None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:11.757778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:11.779536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:11.795498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:11.795943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961a22a-4b93-4373-856d-da991c64adf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:11.796566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961a22a-4b93-4373-856d-da991c64adf2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.796822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.796925 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961a22a-4b93-4373-856d-da991c64adf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.797133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961a22a-4b93-4373-856d-da991c64adf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.797525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.797957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.798839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] RBAC: Adding query filter params (name=network) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:11.799328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:11.801381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:11.806310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 349/714] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:11.811890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:11.883309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961a22a-4b93-4373-856d-da991c64adf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:11.886642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961a22a-4b93-4373-856d-da991c64adf2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:11.922212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669a9dc-cb45-4dcb-9fa7-0028c4b1da2f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:11.958265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77658f-8067-48d0-9097-249f7ef2963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:11.958899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77658f-8067-48d0-9097-249f7ef2963a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:11.959233 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77658f-8067-48d0-9097-249f7ef2963a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:11.959503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77658f-8067-48d0-9097-249f7ef2963a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:11.970859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77658f-8067-48d0-9097-249f7ef2963a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-1b77658f-8067-48d0-9097-249f7ef2963a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:11.976452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:11.977537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:11.978375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 350/715] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:11.979132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77658f-8067-48d0-9097-249f7ef2963a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.021323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 366/716] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:36:12.022463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961a22a-4b93-4373-856d-da991c64adf2 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.030237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:12.049334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:12.089637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77658f-8067-48d0-9097-249f7ef2963a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:12.132473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:12.133255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.133530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.133751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.134412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] RBAC: Adding query filter params (type=network) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:12.134677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:12.136207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:12.136906 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961a22a-4b93-4373-856d-da991c64adf2 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:12.141624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 351/717] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:12.142442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.178411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:12.179529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.180039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.180512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.181370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:12.181844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:12.183640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:12.193875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 367/718] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:12.203699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.322459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014e16b-6897-4333-abfd-8bf669885490 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:12.334489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:12.367795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:12.370061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.370566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.371063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.380583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:12.390511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:12.392370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:12.399797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ba443-a68d-4e74-b10b-7e56c59dceb1 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:12.416234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:12.419584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:12.420455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:12.420455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:12.420455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:12.420455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 352/719] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:12.428110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.481908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:12.482550 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.482825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.483111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.505426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:12.506880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:12.537593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:12.538594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:12.540661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:12.540661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:12.540661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:12.540661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 368/720] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:12.545199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.558272 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e2682-39dd-4988-bff9-5f08872ea094 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:12.599421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:12.600661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.601134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.601619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.603259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:12.603827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:12.605926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:12.624923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 353/721] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:12.626509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.652203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:12.750336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3263b1-3bee-49ae-8102-6592de336126 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:12.762433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f8154-4fbb-473a-a0e2-c19c6c509667 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:12.801008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475897b-df82-4c66-89e4-260df038868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:12.801739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475897b-df82-4c66-89e4-260df038868b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.801858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a205ce-920b-46ab-a394-2f83b198896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:12.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475897b-df82-4c66-89e4-260df038868b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.802289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475897b-df82-4c66-89e4-260df038868b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.802389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a205ce-920b-46ab-a394-2f83b198896a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.802647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a205ce-920b-46ab-a394-2f83b198896a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.803027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a205ce-920b-46ab-a394-2f83b198896a None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.804044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a205ce-920b-46ab-a394-2f83b198896a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:12.804417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a205ce-920b-46ab-a394-2f83b198896a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:12.805812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a205ce-920b-46ab-a394-2f83b198896a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:12.808875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:12.812104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475897b-df82-4c66-89e4-260df038868b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:12.813759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475897b-df82-4c66-89e4-260df038868b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:12.819675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 369/722] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:12.820952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9076c-6ca9-4a79-ae52-0c37d8c94c17 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.821225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9076c-6ca9-4a79-ae52-0c37d8c94c17 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.821907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9076c-6ca9-4a79-ae52-0c37d8c94c17 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.826852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 370/723] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:12.826852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9076c-6ca9-4a79-ae52-0c37d8c94c17 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-f475897b-df82-4c66-89e4-260df038868b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:12.834431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:12.835523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:12.836427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:12.836427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:12.836427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:12.836427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:12.836427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:12.836427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:12.836427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:12.836427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:12.839631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 354/724] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:12.841334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55dcd5-ed4a-45ad-aad1-a7476ecb1bea None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.841799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55dcd5-ed4a-45ad-aad1-a7476ecb1bea None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.842389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55dcd5-ed4a-45ad-aad1-a7476ecb1bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.867711 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-df55dcd5-ed4a-45ad-aad1-a7476ecb1bea None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:12.869659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-df55dcd5-ed4a-45ad-aad1-a7476ecb1bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:12.936623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9076c-6ca9-4a79-ae52-0c37d8c94c17 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:12.955250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 355/725] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:12.956446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55dcd5-ed4a-45ad-aad1-a7476ecb1bea None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:12.973756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:12.974573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:12.975186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:12.975470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:12.976965 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:12.986954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:12.986954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:13.002964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:13.004015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:13.004944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 371/726] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:36:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:13.007911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:13.123183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:13.163406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55dcd5-ed4a-45ad-aad1-a7476ecb1bea None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:13.208102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c4fc9-0417-4a0f-bc8c-0b9ca2305986 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:13.235671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:13.236578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.237025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.237461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.238404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:13.238889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:13.241181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:13.255087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 356/727] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:13.261789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:13.308018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:13.308635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.308916 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.309183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.309919 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:13.310220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:13.312322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:13.342810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 372/728] 10.222.0.94 () {56 vars in 1149 bytes} [Fri Aug 26 15:36:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:13.343704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:13.351992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:13.398590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c105034-eec6-4233-beb0-cefffbefdcf0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:13.447423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:13.448322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.448601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.448873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.449866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=500d2f2bd83d499aa2814c6d7cc71795) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:13.450567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:13.452178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:13.460603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 357/729] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:13 2022] GET /identity/v3/role_assignments?user.id=500d2f2bd83d499aa2814c6d7cc71795&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:13.462113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:13.468139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9411f-4aa8-48a2-8a0f-e3ca3cb71a4b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:13.488043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:13.514626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cdb4b-019a-49e7-a5bd-4cb87ee68c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:13.515416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cdb4b-019a-49e7-a5bd-4cb87ee68c2d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.515802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cdb4b-019a-49e7-a5bd-4cb87ee68c2d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.516252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cdb4b-019a-49e7-a5bd-4cb87ee68c2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.531980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cdb4b-019a-49e7-a5bd-4cb87ee68c2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-b64cdb4b-019a-49e7-a5bd-4cb87ee68c2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:13.538579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:13.539997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:13.540988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:13.543929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 373/730] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:13.549709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cdb4b-019a-49e7-a5bd-4cb87ee68c2d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:13.601868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798ebaa-34ee-476f-aead-ddc9529c5ba8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:13.643481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f413b1-c04d-4215-9094-226978a69530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:13.644259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f413b1-c04d-4215-9094-226978a69530 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.644638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f413b1-c04d-4215-9094-226978a69530 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.645058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f413b1-c04d-4215-9094-226978a69530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.648162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:13.682563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cdb4b-019a-49e7-a5bd-4cb87ee68c2d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:13.725611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f1f05-3c42-455f-92b5-45264309d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:13.726136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f1f05-3c42-455f-92b5-45264309d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.726405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f1f05-3c42-455f-92b5-45264309d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.726670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f1f05-3c42-455f-92b5-45264309d6b9 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.727663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f1f05-3c42-455f-92b5-45264309d6b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:13.727967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f1f05-3c42-455f-92b5-45264309d6b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:13.730023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f1f05-3c42-455f-92b5-45264309d6b9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:13.730861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f413b1-c04d-4215-9094-226978a69530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:13.732884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f413b1-c04d-4215-9094-226978a69530 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:13.740156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 374/731] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:13.741727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390c9a9-63a5-4636-876e-bedaa6bf1a5f None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.742456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390c9a9-63a5-4636-876e-bedaa6bf1a5f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.742743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390c9a9-63a5-4636-876e-bedaa6bf1a5f None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.744810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 375/732] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:13.747906 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390c9a9-63a5-4636-876e-bedaa6bf1a5f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:13.795286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:13.867768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 358/733] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:36:13.869373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6cb66-9739-4cdf-a74d-c6bdbe57c0c4 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.869658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6cb66-9739-4cdf-a74d-c6bdbe57c0c4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.869929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6cb66-9739-4cdf-a74d-c6bdbe57c0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.885364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390c9a9-63a5-4636-876e-bedaa6bf1a5f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:13.896177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3be6cb66-9739-4cdf-a74d-c6bdbe57c0c4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:13.897860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3be6cb66-9739-4cdf-a74d-c6bdbe57c0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:13.934793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:13.936138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:13.936837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:13.937640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:13.941801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:13.959350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:13.963353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:13.986719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:13.988732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:13.990273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:13.990273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:13.990273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:13.994462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 376/734] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:13.995471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:14.005571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 359/735] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:14.007596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6cb66-9739-4cdf-a74d-c6bdbe57c0c4 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:14.126964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:14.144653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f4480-4ac1-48e4-ab91-a0d5d0cecd67 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:14.154667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6cb66-9739-4cdf-a74d-c6bdbe57c0c4 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:14.192672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:14.194244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.194912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.195411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.201740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:14.202355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.202625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.202890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.203568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:14.203889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:14.205209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:14.213020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:14.216835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:14.219841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 360/736] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:14.225115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:14.241320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:14.242258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:14.243549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:14.243549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:14.243549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:14.245635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 377/737] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:14.249878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:14.257329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:14.370652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cfde4-2428-4f9a-b541-233a9a10a868 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:14.427551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a07941-bbb3-4726-9fd7-2b5ba8e95734 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:14.443125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:14.443778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.444052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.444321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] PATH_INFO: `/v3/users/swift` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.454260 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:14.455741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:14.468799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:14.469813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:14.470737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 361/738] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:14.471385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:14.474846 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:14.475803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.476150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.476545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.477507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:14.478110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:14.479751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:14.489709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:14.495258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 378/739] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:14.496170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:14.585196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:14.604560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b7dc9-5a55-484d-aab7-0e4ac11cad2b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:14.612828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec0c89-b88c-4427-99fb-e36d5c3cd502 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:14.641623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6504f7d-b8a3-4928-9d7d-dfb15074d5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:14.642615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6504f7d-b8a3-4928-9d7d-dfb15074d5c1 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.643147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6504f7d-b8a3-4928-9d7d-dfb15074d5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.643511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6504f7d-b8a3-4928-9d7d-dfb15074d5c1 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.644261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6504f7d-b8a3-4928-9d7d-dfb15074d5c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:14.645708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6504f7d-b8a3-4928-9d7d-dfb15074d5c1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:14.650906 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:14.651547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.652217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.652492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.653245 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:14.653547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:14.654977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:14.673432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 379/740] 10.222.0.94 () {56 vars in 1135 bytes} [Fri Aug 26 15:36:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:14.674288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:14.735636 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:14.752816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 362/741] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:36:14 2022] POST /identity/v3/services => generated 251 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:14.754113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d1d6-7bfd-4d87-abc1-93bb315bede5 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.754458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d1d6-7bfd-4d87-abc1-93bb315bede5 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.754785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0d1d6-7bfd-4d87-abc1-93bb315bede5 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.758967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 363/742] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:14.758967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0d1d6-7bfd-4d87-abc1-93bb315bede5 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:14.800242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc11fb0-cd04-4bc9-9834-dbd9557a5c03 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:14.859060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e412292-36c4-4c30-b5cf-54b94281efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:14.859713 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e412292-36c4-4c30-b5cf-54b94281efb0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.859993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e412292-36c4-4c30-b5cf-54b94281efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.860265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e412292-36c4-4c30-b5cf-54b94281efb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.869509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e412292-36c4-4c30-b5cf-54b94281efb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:14.871146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e412292-36c4-4c30-b5cf-54b94281efb0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-3e412292-36c4-4c30-b5cf-54b94281efb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:14.885159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:14.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:14.887030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:14.887030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:14.887030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:14.887030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:14.887030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:14.887030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:14.887030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:14.887030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:14.891102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 380/743] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:14.892052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0d1d6-7bfd-4d87-abc1-93bb315bede5 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:14.892549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad2cd-8f03-4abd-827d-a868855d8eaa None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.892833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad2cd-8f03-4abd-827d-a868855d8eaa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.893096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ad2cd-8f03-4abd-827d-a868855d8eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.916878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-dd0ad2cd-8f03-4abd-827d-a868855d8eaa None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:14.918073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-dd0ad2cd-8f03-4abd-827d-a868855d8eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:14.938133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575d144-bf18-4c60-ba82-d50bc96224b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:14.939089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575d144-bf18-4c60-ba82-d50bc96224b8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:14.939554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575d144-bf18-4c60-ba82-d50bc96224b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:14.940031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575d144-bf18-4c60-ba82-d50bc96224b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:14.958675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:15.010087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 381/744] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:14 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 15:36:15.011501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ad2cd-8f03-4abd-827d-a868855d8eaa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:15.026979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575d144-bf18-4c60-ba82-d50bc96224b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:15.028398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575d144-bf18-4c60-ba82-d50bc96224b8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:15.073368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:15.149634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ad2cd-8f03-4abd-827d-a868855d8eaa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:15.152372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 364/745] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:14 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 15:36:15.158621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575d144-bf18-4c60-ba82-d50bc96224b8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:15.195033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:15.196896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:15.197369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:15.197823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:15.198999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:15.200417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:15.202152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:15.216654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:15.218585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 382/746] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:15.225622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:15.331581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575d144-bf18-4c60-ba82-d50bc96224b8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:15.392752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c52a2-f56b-4523-86ed-f81e3d696c62 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:15.396176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:15.406657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:15.407968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:15.408404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:15.413940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:15.425725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:15.428265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:15.435295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:15.435918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:15.436200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:15.442437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:15.442437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=9e1dd86804944fd1806ec289181b361e) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:15.442437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:15.442437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:15.447928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 383/747] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:15 2022] GET /identity/v3/role_assignments?user.id=9e1dd86804944fd1806ec289181b361e&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:15.448536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:15.449621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:15.450486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:15.450486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:15.450486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:15.450486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 365/748] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:15.450707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:15.456288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:15.596616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:15.605042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74164b0f-7f60-4912-b2a1-6450895d003b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:15.630000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbda04-2586-4682-ba70-398d50937f15 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:15.647787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29438be2-cd95-404b-b783-e445e29ff27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:15.648678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29438be2-cd95-404b-b783-e445e29ff27d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:15.649232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29438be2-cd95-404b-b783-e445e29ff27d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:15.649653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29438be2-cd95-404b-b783-e445e29ff27d None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:15.650454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29438be2-cd95-404b-b783-e445e29ff27d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:15.650913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29438be2-cd95-404b-b783-e445e29ff27d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:15.652491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29438be2-cd95-404b-b783-e445e29ff27d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:15.670470 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 366/749] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:15.670676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b799dd8-6b6a-4a57-97f9-3b922774b8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:15.670676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b799dd8-6b6a-4a57-97f9-3b922774b8b7 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:15.670960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b799dd8-6b6a-4a57-97f9-3b922774b8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:15.671338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b799dd8-6b6a-4a57-97f9-3b922774b8b7 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:15.672107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b799dd8-6b6a-4a57-97f9-3b922774b8b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:15.672736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6dad9-517d-4d11-a123-774f8ff5518c None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:15.673164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6dad9-517d-4d11-a123-774f8ff5518c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:15.673594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6dad9-517d-4d11-a123-774f8ff5518c None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:15.673842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b799dd8-6b6a-4a57-97f9-3b922774b8b7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:15.675579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 367/750] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:15.676510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6dad9-517d-4d11-a123-774f8ff5518c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:15.766768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 384/751] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:36:15 2022] POST /identity/v3/services => generated 240 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:15.768874 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca88115-9e5f-47f3-b440-40cc6bbabfd3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:15.771357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca88115-9e5f-47f3-b440-40cc6bbabfd3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:15.772591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca88115-9e5f-47f3-b440-40cc6bbabfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:15.811259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1ca88115-9e5f-47f3-b440-40cc6bbabfd3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:15.814473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1ca88115-9e5f-47f3-b440-40cc6bbabfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:15.832613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6dad9-517d-4d11-a123-774f8ff5518c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:15.870940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:15.911767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:15.912730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:15.913160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:15.913649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:15.927992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:15.931958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:15.953094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:15.954248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:15.955305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:15.955305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:15.955305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:15.955305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:15.955305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:15.955305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:15.955305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:15.955305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:15.956603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 368/752] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:15.964290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:15.972441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 385/753] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:15.981478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:16.077064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b40e72-d655-45ed-9578-6bb0bdf64634 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:16.116167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:16.117157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:16.117672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:16.118252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:16.119296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:16.119824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:16.121587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:16.136922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 369/754] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:16.143291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca88115-9e5f-47f3-b440-40cc6bbabfd3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:16.272734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca88115-9e5f-47f3-b440-40cc6bbabfd3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:16.287846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:16.315566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:16.316490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:16.317526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:16.317988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:16.319210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=b92a2d92e27d4fe48b42f9e04be92372) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:16.319989 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:16.322699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:16.329791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 386/755] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:16 2022] GET /identity/v3/role_assignments?user.id=b92a2d92e27d4fe48b42f9e04be92372&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:16.434099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:16.475716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127647ad-860f-424b-8112-6d27acd21b4b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:16.525701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4dfbd-e2cf-450d-ae67-7e77cc379f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:16.526355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4dfbd-e2cf-450d-ae67-7e77cc379f12 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:16.526633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4dfbd-e2cf-450d-ae67-7e77cc379f12 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:16.526907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4dfbd-e2cf-450d-ae67-7e77cc379f12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:16.537817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4dfbd-e2cf-450d-ae67-7e77cc379f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-22c4dfbd-e2cf-450d-ae67-7e77cc379f12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:16.544306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:16.545221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:16.546026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 370/756] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:16.553299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:16.582953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:16.666778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e8cc2-062b-4a68-8fd8-9aa04791b044 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:16.714769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:16.715337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:16.715613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:16.715881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:16.716556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:16.716856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:16.718227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:16.727936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 387/757] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:16.736224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4dfbd-e2cf-450d-ae67-7e77cc379f12 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:16.862866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4dfbd-e2cf-450d-ae67-7e77cc379f12 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:16.903970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:16.909062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:16.909700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:16.909979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:16.910255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:16.926663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:16.928167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:16.947657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:16.948646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:16.949517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:16.949517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:16.949517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:16.949517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 371/758] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:16.955968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:17.052188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:17.069828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195e4f-245d-43aa-b23d-f55c8cec09de None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:17.107965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcc2fd-ad78-4204-b61f-f1b9211523a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:17.108580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcc2fd-ad78-4204-b61f-f1b9211523a4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:17.108864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcc2fd-ad78-4204-b61f-f1b9211523a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:17.109195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcc2fd-ad78-4204-b61f-f1b9211523a4 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:17.109844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcc2fd-ad78-4204-b61f-f1b9211523a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:17.110142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcc2fd-ad78-4204-b61f-f1b9211523a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:17.111669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcc2fd-ad78-4204-b61f-f1b9211523a4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:17.124513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 388/759] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:17.132110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:17.215420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb6d8f-7050-4554-93b3-33a374010289 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:17.215745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb6d8f-7050-4554-93b3-33a374010289 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:17.215980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb6d8f-7050-4554-93b3-33a374010289 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:17.217499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 389/760] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:17.224339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b2c02-66de-4555-8146-ba99aec29ae2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:17.224624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b2c02-66de-4555-8146-ba99aec29ae2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:17.224899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b2c02-66de-4555-8146-ba99aec29ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:17.259953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e48b2c02-66de-4555-8146-ba99aec29ae2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:17.261260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e48b2c02-66de-4555-8146-ba99aec29ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:17.279065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2798579-b383-4f27-8508-b7acb7f7e68e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:17.322687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:17.323691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:17.324162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:17.324647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:17.332986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:17.334680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:17.346467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:17.347448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:17.348355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:17.350489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 372/761] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:36:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:17.356530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:17.406423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:17.445724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 390/762] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:17.480641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e741a-f7ad-410f-b20a-6025ec444393 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:17.517560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfe76e-ab45-4c99-956f-6990e2bc1be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:17.518540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfe76e-ab45-4c99-956f-6990e2bc1be8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:17.519077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfe76e-ab45-4c99-956f-6990e2bc1be8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:17.519545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfe76e-ab45-4c99-956f-6990e2bc1be8 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:17.520462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfe76e-ab45-4c99-956f-6990e2bc1be8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:17.520955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfe76e-ab45-4c99-956f-6990e2bc1be8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:17.522563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfe76e-ab45-4c99-956f-6990e2bc1be8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:17.542136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 373/763] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:36:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:17.547900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b2c02-66de-4555-8146-ba99aec29ae2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:17.668231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:17.704116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b2c02-66de-4555-8146-ba99aec29ae2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:17.761959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:17.762607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:17.762877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:17.763170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:17.773409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d49f-19ec-42a4-94ca-8129d61c6dd0 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:17.773715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d49f-19ec-42a4-94ca-8129d61c6dd0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:17.774004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d49f-19ec-42a4-94ca-8129d61c6dd0 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:17.777267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:17.778961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:17.783296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 374/764] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:17.792326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafa195-637d-45ca-9c2e-b11db4a5fe41 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:17.792611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafa195-637d-45ca-9c2e-b11db4a5fe41 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:17.792868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafa195-637d-45ca-9c2e-b11db4a5fe41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:17.800692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:17.801652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:17.802717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:17.802717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:17.802717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:17.804957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 391/765] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:17.813791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:17.816027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3fafa195-637d-45ca-9c2e-b11db4a5fe41 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:17.817332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3fafa195-637d-45ca-9c2e-b11db4a5fe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:17.889455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 375/766] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:17.974153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafa195-637d-45ca-9c2e-b11db4a5fe41 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.031875 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:18.044975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3ae2b-cfe7-4fa4-85a4-5bcf38e554b5 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.093298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:18.110888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafa195-637d-45ca-9c2e-b11db4a5fe41 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.121257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.121880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.122152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.122412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.123085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:18.123380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:18.124760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:18.151928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 392/767] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:18.155342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.167187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.167835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.168113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.168378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.168959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:18.170327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:18.183642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:18.184732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:18.185572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:18.187351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 376/768] 10.222.0.94 () {56 vars in 1129 bytes} [Fri Aug 26 15:36:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:18.192718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.297978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83be36a-2b10-4ac9-8629-79ec1329837d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.335903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.336558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.336829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.337092 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.340750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:18.344580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:18.345995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:18.349175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fec3ae-72c8-41bc-8352-e0a9af230fcf None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:18.360784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:18.361726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:18.362597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 393/769] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:18.363112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.394086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.395122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.395604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.396089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.396928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:18.397448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:18.399211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:18.409123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 377/770] 10.222.0.94 () {56 vars in 1124 bytes} [Fri Aug 26 15:36:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:18.410473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.422788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:18.505997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb6df7-b1b7-4da9-9147-3a55f660c085 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.537162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f369b0-431b-42ac-a070-c5de07454fc0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.551191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3f5ca-d5ee-4ee5-bfca-4ce7d780d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.552223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3f5ca-d5ee-4ee5-bfca-4ce7d780d10b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.552526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3f5ca-d5ee-4ee5-bfca-4ce7d780d10b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.552791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3f5ca-d5ee-4ee5-bfca-4ce7d780d10b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.567958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3f5ca-d5ee-4ee5-bfca-4ce7d780d10b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-45b3f5ca-d5ee-4ee5-bfca-4ce7d780d10b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:18.573187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:18.574155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:18.575004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:18.579177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 394/771] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:18.580077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3f5ca-d5ee-4ee5-bfca-4ce7d780d10b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.588246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112dd7c-b994-471c-92b1-eb2d9b231d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.588894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112dd7c-b994-471c-92b1-eb2d9b231d2e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.589172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112dd7c-b994-471c-92b1-eb2d9b231d2e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.589439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112dd7c-b994-471c-92b1-eb2d9b231d2e None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.590147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112dd7c-b994-471c-92b1-eb2d9b231d2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:18.590443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112dd7c-b994-471c-92b1-eb2d9b231d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:18.591877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112dd7c-b994-471c-92b1-eb2d9b231d2e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:18.606796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 378/772] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:18.608080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241697b-3f32-4726-a0be-4db92e54018a None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.608363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241697b-3f32-4726-a0be-4db92e54018a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.608634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241697b-3f32-4726-a0be-4db92e54018a None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.610018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 379/773] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:18.610557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241697b-3f32-4726-a0be-4db92e54018a None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.642011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:18.700254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3f5ca-d5ee-4ee5-bfca-4ce7d780d10b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.707262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:18.733143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241697b-3f32-4726-a0be-4db92e54018a None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.734111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.734723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.735030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.735446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.735943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:18.736238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:18.737612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:18.745036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 395/774] 10.222.0.94 () {56 vars in 1124 bytes} [Fri Aug 26 15:36:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:18.745813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.768809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db4bb1-71e3-4b3e-ab0d-4e577ef481b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.769414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4bb1-71e3-4b3e-ab0d-4e577ef481b1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.769691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4bb1-71e3-4b3e-ab0d-4e577ef481b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.769959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4bb1-71e3-4b3e-ab0d-4e577ef481b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.770623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db4bb1-71e3-4b3e-ab0d-4e577ef481b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:18.770915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db4bb1-71e3-4b3e-ab0d-4e577ef481b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:18.772303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db4bb1-71e3-4b3e-ab0d-4e577ef481b1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:18.780865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 380/775] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:18.782045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36575549-dcda-4cde-9757-ea999255f6e0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.782330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36575549-dcda-4cde-9757-ea999255f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.782605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36575549-dcda-4cde-9757-ea999255f6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.802896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-36575549-dcda-4cde-9757-ea999255f6e0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:18.804144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-36575549-dcda-4cde-9757-ea999255f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:18.848483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d25b-2e0f-4af6-94e1-e8ac0c098d90 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:18.873075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 381/776] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:18.874321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e265b41-a966-4be4-8c2c-83eac7a5cbb8 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.874607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e265b41-a966-4be4-8c2c-83eac7a5cbb8 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.874886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e265b41-a966-4be4-8c2c-83eac7a5cbb8 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.876157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 382/777] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:18.876851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e265b41-a966-4be4-8c2c-83eac7a5cbb8 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:18.887170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ea5f8-fe72-4bfe-9b13-2b9f03f4d488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:18.887976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ea5f8-fe72-4bfe-9b13-2b9f03f4d488 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:18.888253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ea5f8-fe72-4bfe-9b13-2b9f03f4d488 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:18.888520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ea5f8-fe72-4bfe-9b13-2b9f03f4d488 None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/500d2f2bd83d499aa2814c6d7cc71795/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:18.890448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ea5f8-fe72-4bfe-9b13-2b9f03f4d488 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=500d2f2bd83d499aa2814c6d7cc71795, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:18.891938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ea5f8-fe72-4bfe-9b13-2b9f03f4d488 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:18.898625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:18.964517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 396/778] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:36:18 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/500d2f2bd83d499aa2814c6d7cc71795/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:36:18.965277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ea5f8-fe72-4bfe-9b13-2b9f03f4d488 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.006316 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:19.008042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e265b41-a966-4be4-8c2c-83eac7a5cbb8 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.060614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07063bf0-d5eb-4a03-8192-23f6efc4abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.061624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07063bf0-d5eb-4a03-8192-23f6efc4abf9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.062109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07063bf0-d5eb-4a03-8192-23f6efc4abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.062615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07063bf0-d5eb-4a03-8192-23f6efc4abf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.063595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07063bf0-d5eb-4a03-8192-23f6efc4abf9 None admin] RBAC: Adding query filter params (interface=public, service_id=514c820e6e994c9aa6c9bc1cd7354572, region_id=RegionOne) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:19.064118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07063bf0-d5eb-4a03-8192-23f6efc4abf9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:19.066680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07063bf0-d5eb-4a03-8192-23f6efc4abf9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:19.078388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 383/779] 10.222.0.94 () {56 vars in 1251 bytes} [Fri Aug 26 15:36:18 2022] GET /identity/v3/endpoints?service_id=514c820e6e994c9aa6c9bc1cd7354572&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:19.080195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a805f-1a73-4e3b-b722-836c06e6f66d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.080719 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a805f-1a73-4e3b-b722-836c06e6f66d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.081198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a805f-1a73-4e3b-b722-836c06e6f66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.103424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ea5f8-fe72-4bfe-9b13-2b9f03f4d488 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.105034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-351a805f-1a73-4e3b-b722-836c06e6f66d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:19.106511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-351a805f-1a73-4e3b-b722-836c06e6f66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:19.149764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.150516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.150787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.151082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.165852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:19.168179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:19.186238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:19.187391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:19.188292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:19.188292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:19.188292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:19.188292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 397/780] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:19.188292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.194016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 384/781] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:19.194890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a805f-1a73-4e3b-b722-836c06e6f66d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.206140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:19.280259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:19.300035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff03cde-f6b5-44ed-9cdb-1f8a4161d155 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.313534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a805f-1a73-4e3b-b722-836c06e6f66d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.337838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77591e-22f6-448d-8f09-b9d69597c4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.338507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77591e-22f6-448d-8f09-b9d69597c4d0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.338782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77591e-22f6-448d-8f09-b9d69597c4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.339070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77591e-22f6-448d-8f09-b9d69597c4d0 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.339692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77591e-22f6-448d-8f09-b9d69597c4d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:19.341123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77591e-22f6-448d-8f09-b9d69597c4d0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:19.347711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 398/782] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/services => generated 772 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:19.348679 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77591e-22f6-448d-8f09-b9d69597c4d0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.475644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77591e-22f6-448d-8f09-b9d69597c4d0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.486385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003531ef-665f-4374-95c8-f66540700294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.487661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003531ef-665f-4374-95c8-f66540700294 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.488325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003531ef-665f-4374-95c8-f66540700294 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.489066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003531ef-665f-4374-95c8-f66540700294 None admin] PATH_INFO: `/v3/services/network` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.489954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003531ef-665f-4374-95c8-f66540700294 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:19.491844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003531ef-665f-4374-95c8-f66540700294 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-003531ef-665f-4374-95c8-f66540700294 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:19.502971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:19.504035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:19.504844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:19.508196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 385/783] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/services/network => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:19.520677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003531ef-665f-4374-95c8-f66540700294 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.525452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.526162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.526529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.526894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.527604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:19.527903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:19.529420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:19.543747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 399/784] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:19.552309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.563323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:19.625639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:19.664286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003531ef-665f-4374-95c8-f66540700294 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.685731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c573c-eab3-4518-82df-ff70896c38ac None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.703931 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.704549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.704816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.705085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.705712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] RBAC: Adding query filter params (name=network) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:19.706002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:19.707388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:19.713718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 386/785] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:19.714529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.739512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.741171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.741644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.743327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.754607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:19.757444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:19.773012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:19.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:19.775873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:19.776389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 400/786] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:19.781825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.844754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81232b26-3c30-4785-a2bf-3c1549b38b23 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.863281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:19.893695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4c76b-04a1-4611-9a2c-45937348d2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.894864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4c76b-04a1-4611-9a2c-45937348d2c3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.895664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4c76b-04a1-4611-9a2c-45937348d2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.896418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4c76b-04a1-4611-9a2c-45937348d2c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.900195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:19.902196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c035ba-8dd4-4a3b-8bfb-d1e7031165e9 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:19.916975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4c76b-04a1-4611-9a2c-45937348d2c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e7d4c76b-04a1-4611-9a2c-45937348d2c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:19.923991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:19.925006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:19.925836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:19.933733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 387/787] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:19.933733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4c76b-04a1-4611-9a2c-45937348d2c3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:19.957100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:19.958066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:19.958788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:19.959258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:19.960069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] RBAC: Adding query filter params (type=network) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:19.960541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:19.963077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:19.968784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 401/788] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:19.969951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.069208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4c76b-04a1-4611-9a2c-45937348d2c3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:20.100416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b1b3c-05c1-407c-b88e-ae0144419a8a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:20.120332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:20.121385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:20.121676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:20.121944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:20.122694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:20.123020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:20.125084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:20.134586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:20.141848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:20.142671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:20.142982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:20.143254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:20.148429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:20.148429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:20.148429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:20.153562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 388/789] 10.222.0.94 () {56 vars in 1149 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:20.155003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.157530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 402/790] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:20.163971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.220422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:20.276418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0e183-293f-4e20-98fc-d7c1cb317f9a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:20.304558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c4d74-1f2f-49dc-a9b7-b9b0ac18d057 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:20.319252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:20.320229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:20.321568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:20.322048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:20.323425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] RBAC: Adding query filter params (interface=public, service_id=600561a899d6436ea35969f2bc57a813, region_id=RegionOne) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:20.324205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:20.325956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:20.333397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 389/791] 10.222.0.94 () {56 vars in 1251 bytes} [Fri Aug 26 15:36:20 2022] GET /identity/v3/endpoints?service_id=600561a899d6436ea35969f2bc57a813&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:20.334840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.345526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:20.346220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:20.346494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:20.346761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:20.358304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:20.359695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:20.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:20.377396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:20.378271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:20.378271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:20.378271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:20.382525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 403/792] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:20.383836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.442342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:20.473600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182602a-47e2-4dd3-926f-44ff52b1e8b1 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:20.499950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:20.515400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73a02-dc36-4313-ba51-f6d6a92e743e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:20.530274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:20.531646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:20.532164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:20.532637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:20.550483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:20.552473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:20.559153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec98638-ccf3-4db9-9f8e-c2cad2818b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:20.559927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec98638-ccf3-4db9-9f8e-c2cad2818b1c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:20.560290 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec98638-ccf3-4db9-9f8e-c2cad2818b1c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:20.560614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec98638-ccf3-4db9-9f8e-c2cad2818b1c None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:20.563867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec98638-ccf3-4db9-9f8e-c2cad2818b1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:20.563867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec98638-ccf3-4db9-9f8e-c2cad2818b1c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:20.569856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 404/793] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:36:20 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:20.570480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec98638-ccf3-4db9-9f8e-c2cad2818b1c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:20.589234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:20.590310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:20.591206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:20.591206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:20.591206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:20.593596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 390/794] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:20.599998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.749894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c52bd-4b6f-4631-bbf2-1912f989de20 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:20.785674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec98638-ccf3-4db9-9f8e-c2cad2818b1c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:20.791300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:20.791300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:20.791724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:20.792047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:20.792767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:20.793128 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:20.794568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:20.800162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:20.809071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 391/795] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:20.818734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.834988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:20.834988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:20.834988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:20.834988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:20.835443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:20.835443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:20.835443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:20.849684 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 405/796] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:20.855945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:20.868174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:21.017444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeb12f-f62a-422b-94d8-c399361ecbfe None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:21.027578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436a032-6f02-4b2e-b310-ccebcd7d2e1d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:21.112618 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:21.113276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.113549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.113816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] PATH_INFO: `/v3/users/swift` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.129875 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:21.131399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:21.154832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:21.155852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:21.156772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:21.161755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 392/797] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:21.164371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:21.189573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182ad52-40d6-418d-97d9-c8b35f25cd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:21.190619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182ad52-40d6-418d-97d9-c8b35f25cd53 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.190619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182ad52-40d6-418d-97d9-c8b35f25cd53 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.190880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182ad52-40d6-418d-97d9-c8b35f25cd53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.201000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182ad52-40d6-418d-97d9-c8b35f25cd53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:21.202640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182ad52-40d6-418d-97d9-c8b35f25cd53 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-b182ad52-40d6-418d-97d9-c8b35f25cd53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:21.219144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:21.220105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:21.220958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 406/798] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:21.221933 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e4be0-24ee-4470-a892-fab149d215ff None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.222097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e4be0-24ee-4470-a892-fab149d215ff None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.222342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e4be0-24ee-4470-a892-fab149d215ff None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.229133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 407/799] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:21.229133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e4be0-24ee-4470-a892-fab149d215ff None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:21.263491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:21.278053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:21.292001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e9a00-462a-4f8c-b067-f88ed071874f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:21.331393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68c28e-c9d6-4e63-a8f0-b21f248ef300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:21.332442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68c28e-c9d6-4e63-a8f0-b21f248ef300 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.332918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68c28e-c9d6-4e63-a8f0-b21f248ef300 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.333400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68c28e-c9d6-4e63-a8f0-b21f248ef300 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.334495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68c28e-c9d6-4e63-a8f0-b21f248ef300 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:21.335134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68c28e-c9d6-4e63-a8f0-b21f248ef300 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:21.337629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68c28e-c9d6-4e63-a8f0-b21f248ef300 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:21.350499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e4be0-24ee-4470-a892-fab149d215ff None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:21.359484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 393/800] 10.222.0.94 () {56 vars in 1135 bytes} [Fri Aug 26 15:36:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:21.361031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df4f94-f111-4a9f-8247-47921729ce2b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.361535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df4f94-f111-4a9f-8247-47921729ce2b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.361992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df4f94-f111-4a9f-8247-47921729ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.383279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e9df4f94-f111-4a9f-8247-47921729ce2b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:21.384937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e9df4f94-f111-4a9f-8247-47921729ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:21.394421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:21.395073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.395353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.395619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.396307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:21.396903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:21.398139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:21.417759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 408/801] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:21.418808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:21.462549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 394/802] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:21.463760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df4f94-f111-4a9f-8247-47921729ce2b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:21.557566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:21.583580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:21.588580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df4f94-f111-4a9f-8247-47921729ce2b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:21.599636 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4a63a-05a9-444c-ad6a-8de2a2bf8714 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:21.626046 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba2807-65d5-419a-810d-64a7a9aadb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:21.627127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba2807-65d5-419a-810d-64a7a9aadb67 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.627611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba2807-65d5-419a-810d-64a7a9aadb67 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.628073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba2807-65d5-419a-810d-64a7a9aadb67 None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/9e1dd86804944fd1806ec289181b361e/roles/b8b6b507d5284b84a25895b280b89cbd` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.643729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba2807-65d5-419a-810d-64a7a9aadb67 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=9e1dd86804944fd1806ec289181b361e, role_id=b8b6b507d5284b84a25895b280b89cbd)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:21.645368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f21b04-ee38-487e-b613-2b8865f70e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:21.645711 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba2807-65d5-419a-810d-64a7a9aadb67 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:21.645996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f21b04-ee38-487e-b613-2b8865f70e94 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.646269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f21b04-ee38-487e-b613-2b8865f70e94 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.646531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f21b04-ee38-487e-b613-2b8865f70e94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.660765 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f21b04-ee38-487e-b613-2b8865f70e94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:21.662263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f21b04-ee38-487e-b613-2b8865f70e94 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-66f21b04-ee38-487e-b613-2b8865f70e94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:21.677833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:21.678804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:21.680799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:21.680799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:21.680799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:21.680799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 409/803] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:21.686961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8542f5-7363-44fa-bdf7-9362b80dedc7 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.686961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8542f5-7363-44fa-bdf7-9362b80dedc7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.686961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8542f5-7363-44fa-bdf7-9362b80dedc7 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.690138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 410/804] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:21.690138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8542f5-7363-44fa-bdf7-9362b80dedc7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:21.783290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 395/805] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:36:21 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/9e1dd86804944fd1806ec289181b361e/roles/b8b6b507d5284b84a25895b280b89cbd => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:36:21.785270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750a3c-1b7f-4ab0-b47f-255014a1ee95 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.785586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750a3c-1b7f-4ab0-b47f-255014a1ee95 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.785846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750a3c-1b7f-4ab0-b47f-255014a1ee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.802650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8542f5-7363-44fa-bdf7-9362b80dedc7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:21.825749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-77750a3c-1b7f-4ab0-b47f-255014a1ee95 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:21.827095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-77750a3c-1b7f-4ab0-b47f-255014a1ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:21.859200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:21.860448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:21.860883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:21.861628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:21.863020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:21.863494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:21.865899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:21.879608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:21.885140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 411/806] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:21.886530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:21.912460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 396/807] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:21.913624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77750a3c-1b7f-4ab0-b47f-255014a1ee95 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:21.985382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:22.033029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77750a3c-1b7f-4ab0-b47f-255014a1ee95 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:22.052147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22622d-1775-4d7b-b292-47efb7d9e011 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:22.083918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:22.084578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.084867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.085130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.089520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79725e7-5ff1-406b-8d3d-fcc871e85393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:22.090317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79725e7-5ff1-406b-8d3d-fcc871e85393 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.091494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79725e7-5ff1-406b-8d3d-fcc871e85393 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.091903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79725e7-5ff1-406b-8d3d-fcc871e85393 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.101052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:22.103948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:22.114548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79725e7-5ff1-406b-8d3d-fcc871e85393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-d79725e7-5ff1-406b-8d3d-fcc871e85393 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:22.120693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:22.121726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:22.122710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:22.129057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:22.130185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:22.131277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:22.131277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:22.131277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:22.131277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:22.131277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:22.131277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:22.131277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:22.131277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:22.134232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 412/808] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:22.135034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 397/809] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:22.138615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79725e7-5ff1-406b-8d3d-fcc871e85393 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:22.140126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:22.239253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:22.293600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c662f5-8ab7-46c1-b183-14f223078818 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:22.298273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79725e7-5ff1-406b-8d3d-fcc871e85393 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:22.312920 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:22.332712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:22.333319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.333594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.333863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.334609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:22.334850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:22.336927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:22.346269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:22.347549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.348068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.348527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.350252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:22.350820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:22.352446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:22.354992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 398/810] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:22.356050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:22.364169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 413/811] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:22.365638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:22.470984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73423d-95fe-4846-a559-0981482bdf49 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:22.476435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a92c28-432e-4e4d-88e6-746ad9c7d564 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:22.503059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:22.511048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:22.511781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.512155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.512456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.513115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:22.514569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:22.516790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aded35e-82b9-4ee0-8737-ad2d7399ed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:22.517686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded35e-82b9-4ee0-8737-ad2d7399ed11 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.518172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded35e-82b9-4ee0-8737-ad2d7399ed11 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.518631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded35e-82b9-4ee0-8737-ad2d7399ed11 None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/b92a2d92e27d4fe48b42f9e04be92372/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.520961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aded35e-82b9-4ee0-8737-ad2d7399ed11 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=b92a2d92e27d4fe48b42f9e04be92372, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:22.522611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aded35e-82b9-4ee0-8737-ad2d7399ed11 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:22.525130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:22.526102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 15:36:22.526982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:22.531277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 399/812] 10.222.0.94 () {56 vars in 1129 bytes} [Fri Aug 26 15:36:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:22.532311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:22.596105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:22.623472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 414/813] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:36:22 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/b92a2d92e27d4fe48b42f9e04be92372/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:36:22.625760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aded35e-82b9-4ee0-8737-ad2d7399ed11 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:22.715746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbd3b0-b7b1-4c9e-aec1-681e2710d07b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:22.793356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:22.794883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.795398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.795872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.810642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:22.812135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:22.828041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:22.829537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:22.830715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:22.830715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:22.830715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:22.830715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 400/814] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:22.833113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aded35e-82b9-4ee0-8737-ad2d7399ed11 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:22.839030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:22.879214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061f063-f936-4f40-b3a4-f4fe1ee49f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:22.880244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f063-f936-4f40-b3a4-f4fe1ee49f4b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.881468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f063-f936-4f40-b3a4-f4fe1ee49f4b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.881744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f063-f936-4f40-b3a4-f4fe1ee49f4b None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.882416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061f063-f936-4f40-b3a4-f4fe1ee49f4b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:22.882720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061f063-f936-4f40-b3a4-f4fe1ee49f4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:22.884176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061f063-f936-4f40-b3a4-f4fe1ee49f4b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:22.893799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 415/815] 10.222.0.94 () {56 vars in 1124 bytes} [Fri Aug 26 15:36:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:22.893799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f21be-5e67-411a-9381-7115af2176fa None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:22.893799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f21be-5e67-411a-9381-7115af2176fa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:22.893799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f21be-5e67-411a-9381-7115af2176fa None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:22.894182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 416/816] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:22.896831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f21be-5e67-411a-9381-7115af2176fa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:22.961936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:22.971258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:22.982527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28d88d-c32a-43fa-8eb5-3d82c63fa766 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:23.022520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f21be-5e67-411a-9381-7115af2176fa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:23.028047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a677f49-ac92-47ad-a84a-066ddb399267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:23.028747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677f49-ac92-47ad-a84a-066ddb399267 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:23.028963 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677f49-ac92-47ad-a84a-066ddb399267 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:23.029239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677f49-ac92-47ad-a84a-066ddb399267 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:23.029915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a677f49-ac92-47ad-a84a-066ddb399267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:23.030217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a677f49-ac92-47ad-a84a-066ddb399267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:23.031760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a677f49-ac92-47ad-a84a-066ddb399267 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:23.049842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 401/817] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:23.051549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7fc51-b5f9-4e95-bc1a-3c9e071647d9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:23.053055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7fc51-b5f9-4e95-bc1a-3c9e071647d9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:23.053510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7fc51-b5f9-4e95-bc1a-3c9e071647d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:23.076909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:23.077621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:23.077973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:23.078314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:23.079224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:23.079599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:23.080477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-77f7fc51-b5f9-4e95-bc1a-3c9e071647d9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:23.081113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:23.082161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-77f7fc51-b5f9-4e95-bc1a-3c9e071647d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:23.088334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 417/818] 10.222.0.94 () {56 vars in 1124 bytes} [Fri Aug 26 15:36:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:23.089313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:23.167274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 402/819] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:23.168659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7fc51-b5f9-4e95-bc1a-3c9e071647d9 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:23.214065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91053336-9c02-42fa-bd69-014a8869d09c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:23.225474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:23.252869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:23.258942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:23.258942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:23.258942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:23.264070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:23.265665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:23.278103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:23.279200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:23.280135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 418/820] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:36:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:23.284688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:23.328934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:23.348715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7fc51-b5f9-4e95-bc1a-3c9e071647d9 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:23.417799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba755b-40da-4f29-8c96-6f2d519d7598 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:23.424897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0074050-4369-4af9-81d0-bb2a8126e338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:23.425839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074050-4369-4af9-81d0-bb2a8126e338 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:23.426319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074050-4369-4af9-81d0-bb2a8126e338 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:23.426785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074050-4369-4af9-81d0-bb2a8126e338 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:23.427683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0074050-4369-4af9-81d0-bb2a8126e338 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:23.429258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0074050-4369-4af9-81d0-bb2a8126e338 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:23.456849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:23.457465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:23.457742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:23.458007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:23.459370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:23.459677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:23.461710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:23.482885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 419/821] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:36:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:23.492196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:23.524563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:23.590834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 403/822] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:36:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:23.591916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0074050-4369-4af9-81d0-bb2a8126e338 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:23.651002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd17cd-4cf5-4099-b039-bb824a00c352 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:23.723963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff414a-3ff5-406d-af82-deb75fa2dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:23.724505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff414a-3ff5-406d-af82-deb75fa2dac3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:23.724836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff414a-3ff5-406d-af82-deb75fa2dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:23.725120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff414a-3ff5-406d-af82-deb75fa2dac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:23.797475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0074050-4369-4af9-81d0-bb2a8126e338 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:23.806983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff414a-3ff5-406d-af82-deb75fa2dac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:23.808466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff414a-3ff5-406d-af82-deb75fa2dac3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:23.848411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:23.849067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:23.849603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:23.850052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:23.850052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] PATH_INFO: `/v3/services/network` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:23.850843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:23.854635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:23.865501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:23.877775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:23.878593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 404/823] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:23 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:23.879395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:23.879395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:23.936314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 420/824] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:36:23.942651 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff414a-3ff5-406d-af82-deb75fa2dac3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:24.029150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304b9ee-707c-4cd4-bb11-c1a22e91466d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:24.065922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba79af3-cde2-47c4-978f-e37849994a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:24.066543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79af3-cde2-47c4-978f-e37849994a36 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.066819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79af3-cde2-47c4-978f-e37849994a36 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.067133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79af3-cde2-47c4-978f-e37849994a36 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.067774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba79af3-cde2-47c4-978f-e37849994a36 None admin] RBAC: Adding query filter params (name=network) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:24.068073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba79af3-cde2-47c4-978f-e37849994a36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:24.068194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff414a-3ff5-406d-af82-deb75fa2dac3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:24.069447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba79af3-cde2-47c4-978f-e37849994a36 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:24.075129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 405/825] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:24.076434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8871e-3f11-4793-a127-74aab4ffb202 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.076720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8871e-3f11-4793-a127-74aab4ffb202 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.076989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8871e-3f11-4793-a127-74aab4ffb202 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.078476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 406/826] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:24.081421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8871e-3f11-4793-a127-74aab4ffb202 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:24.110441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:24.120379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d9369-10f3-472c-935f-5e45603ed797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:24.121362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9369-10f3-472c-935f-5e45603ed797 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.124747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9369-10f3-472c-935f-5e45603ed797 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.124747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d9369-10f3-472c-935f-5e45603ed797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.137892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d9369-10f3-472c-935f-5e45603ed797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:24.140193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d9369-10f3-472c-935f-5e45603ed797 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-506d9369-10f3-472c-935f-5e45603ed797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:24.159122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:24.160310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:24.161385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:24.161385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:24.161385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:24.167562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:24.167835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 421/827] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:24.169289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ba86-5922-4cc8-8630-92b4a2efb192 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.169770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ba86-5922-4cc8-8630-92b4a2efb192 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.170241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ba86-5922-4cc8-8630-92b4a2efb192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.190165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cf74ba86-5922-4cc8-8630-92b4a2efb192 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:24.191681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cf74ba86-5922-4cc8-8630-92b4a2efb192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:24.212331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8871e-3f11-4793-a127-74aab4ffb202 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:24.256784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:24.257847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.258404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.258948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.259887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] RBAC: Adding query filter params (type=network) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:24.260405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:24.262084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:24.268261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 407/828] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:24.269446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:24.272295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 422/829] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:24.288038 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74ba86-5922-4cc8-8630-92b4a2efb192 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:24.383734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700fa35-ebfd-428d-9519-85898ffed44f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:24.415908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:24.422590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:24.423461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.432825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.432825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.432825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:24.432825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:24.432825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:24.439873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 408/830] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:24.447601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:24.502484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74ba86-5922-4cc8-8630-92b4a2efb192 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:24.520621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:24.587324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a4944-23bd-410c-8e53-8b5ac82c6064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:24.588018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a4944-23bd-410c-8e53-8b5ac82c6064 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.588301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a4944-23bd-410c-8e53-8b5ac82c6064 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.588571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a4944-23bd-410c-8e53-8b5ac82c6064 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.589246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a4944-23bd-410c-8e53-8b5ac82c6064 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:24.590631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a4944-23bd-410c-8e53-8b5ac82c6064 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:24.641756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642eeb7-4672-4f6d-b644-559ebc7eeba4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:24.680945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:24.681727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.682059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.682331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.690510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:24.691986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:24.706951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 423/831] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:36:24 2022] POST /identity/v3/endpoints => generated 337 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:36:24.706951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a4944-23bd-410c-8e53-8b5ac82c6064 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:24.732061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:24.733018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:24.733876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:24.733876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:24.733876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:24.733876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:24.733876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:24.733876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:24.733876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:24.733876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:24.734782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 409/832] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:24.740052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:24.808675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:24.895509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a5ea0-e689-4060-b171-0bf5711f7bbf None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:24.929698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a4944-23bd-410c-8e53-8b5ac82c6064 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:24.943601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:24.944233 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.944510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.944778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:24.946097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:24.946432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:24.948139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:24.966176 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 410/833] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:24.972558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:24.988276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a2eae-2d5b-4923-9340-31ab497d82b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:24.988276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a2eae-2d5b-4923-9340-31ab497d82b2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:24.988276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a2eae-2d5b-4923-9340-31ab497d82b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:24.988276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a2eae-2d5b-4923-9340-31ab497d82b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.009659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a2eae-2d5b-4923-9340-31ab497d82b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-6e8a2eae-2d5b-4923-9340-31ab497d82b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:25.017588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:25.018519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:25.019434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 424/834] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:36:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:25.020121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476259-82a0-4055-bec7-17eb7b1a3782 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.020799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476259-82a0-4055-bec7-17eb7b1a3782 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.020799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476259-82a0-4055-bec7-17eb7b1a3782 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.021958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 425/835] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:25.026899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476259-82a0-4055-bec7-17eb7b1a3782 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:25.051339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:25.113763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f16d26-f736-4be4-b766-fa06847c4642 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:25.126845 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:25.151259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476259-82a0-4055-bec7-17eb7b1a3782 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:25.152914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254273-1c72-4975-b809-393823bd6a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:25.153964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254273-1c72-4975-b809-393823bd6a63 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.154237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254273-1c72-4975-b809-393823bd6a63 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.154505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254273-1c72-4975-b809-393823bd6a63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.155253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be254273-1c72-4975-b809-393823bd6a63 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=500d2f2bd83d499aa2814c6d7cc71795) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:25.155558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be254273-1c72-4975-b809-393823bd6a63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:25.157009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be254273-1c72-4975-b809-393823bd6a63 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:25.163343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 411/836] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:24 2022] GET /identity/v3/role_assignments?user.id=500d2f2bd83d499aa2814c6d7cc71795&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:25.164931 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedbed-aa90-431e-9a41-483017d465d4 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.165206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedbed-aa90-431e-9a41-483017d465d4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.165468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedbed-aa90-431e-9a41-483017d465d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.188736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-9cdedbed-aa90-431e-9a41-483017d465d4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:25.190468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9cdedbed-aa90-431e-9a41-483017d465d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:25.200483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:25.202229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.202709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.203226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.204547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:25.205495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:25.207496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:25.220684 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 426/837] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:36:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:25.226511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:25.282634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 412/838] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:25.321256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:25.342431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07061dab-e44d-4c19-9a79-e67e01875c56 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:25.381391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:25.382026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.382300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.382566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.393749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:25.395240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:25.410706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:25.411728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:25.412640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:25.412640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:25.412640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:25.412640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 427/839] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:25.419129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdedbed-aa90-431e-9a41-483017d465d4 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:25.535235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdedbed-aa90-431e-9a41-483017d465d4 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:25.537651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:25.558139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:25.587505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:25.588124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.588388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.588647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.589293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:25.589583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:25.590981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:25.603699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 413/840] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:25.609444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:25.682484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f944b-c891-43bf-ab5a-f9a5ce811880 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:25.715836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:25.721008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adeebf-bbc3-4f9f-b253-6c0f52d1ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:25.721648 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adeebf-bbc3-4f9f-b253-6c0f52d1ea3d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.730625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adeebf-bbc3-4f9f-b253-6c0f52d1ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.730625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adeebf-bbc3-4f9f-b253-6c0f52d1ea3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.734469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adeebf-bbc3-4f9f-b253-6c0f52d1ea3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-b7adeebf-bbc3-4f9f-b253-6c0f52d1ea3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:25.739312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:25.740566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:25.741381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 428/841] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:25.747825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adeebf-bbc3-4f9f-b253-6c0f52d1ea3d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:25.749241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0b56a-3a15-4081-879b-dcb4ae5b2f72 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:25.796080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:25.796721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.797888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.798184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.809306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:25.811072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:25.827101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:25.828624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:25.830009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:25.833090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 414/842] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:36:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:25.834146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:25.870182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:25.892357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adeebf-bbc3-4f9f-b253-6c0f52d1ea3d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:25.933466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b70b4-b3d0-475a-9b41-0b2191c05164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:25.934290 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b70b4-b3d0-475a-9b41-0b2191c05164 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.934568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b70b4-b3d0-475a-9b41-0b2191c05164 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.934832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b70b4-b3d0-475a-9b41-0b2191c05164 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.936010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b70b4-b3d0-475a-9b41-0b2191c05164 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:25.936010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b70b4-b3d0-475a-9b41-0b2191c05164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:25.937187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b70b4-b3d0-475a-9b41-0b2191c05164 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:25.950954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 429/843] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:25.950954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2214-9bf2-42c1-b028-1a1ac26f9a42 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:25.950954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2214-9bf2-42c1-b028-1a1ac26f9a42 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:25.950954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2214-9bf2-42c1-b028-1a1ac26f9a42 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:25.950954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 430/844] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:25.954875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c2214-9bf2-42c1-b028-1a1ac26f9a42 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:25.973373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ff4a7-aeb4-4bc6-bf91-8aeea956c94a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.012132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112fb41-6266-4498-a1c7-5ed0d571bcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.012736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112fb41-6266-4498-a1c7-5ed0d571bcdd None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.013016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112fb41-6266-4498-a1c7-5ed0d571bcdd None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.013286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112fb41-6266-4498-a1c7-5ed0d571bcdd None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.014019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112fb41-6266-4498-a1c7-5ed0d571bcdd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:26.014379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112fb41-6266-4498-a1c7-5ed0d571bcdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.015833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112fb41-6266-4498-a1c7-5ed0d571bcdd None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.021823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:26.035604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 415/845] 10.222.0.94 () {56 vars in 1149 bytes} [Fri Aug 26 15:36:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:26.036672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6c9f8-bd8e-4016-8063-2e6a1a6910af None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.036953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6c9f8-bd8e-4016-8063-2e6a1a6910af None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.037225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6c9f8-bd8e-4016-8063-2e6a1a6910af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.056497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-89a6c9f8-bd8e-4016-8063-2e6a1a6910af None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:26.057733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-89a6c9f8-bd8e-4016-8063-2e6a1a6910af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:26.079261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c2214-9bf2-42c1-b028-1a1ac26f9a42 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.119881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.120665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.121003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.121306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.132749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.134838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.146970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 416/846] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:26.146970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6c9f8-bd8e-4016-8063-2e6a1a6910af None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:26.154910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:26.156112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:26.157109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:26.157109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:26.157109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:26.161085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 431/847] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:26.161974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.170781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:26.289314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a9da9-8a9b-4edb-95a9-1eaac4f959db None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.297396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:26.298904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6c9f8-bd8e-4016-8063-2e6a1a6910af None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.327855 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.328745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.329158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.329569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.330382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:26.330829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.332465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.343800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538ad33-9019-4e11-b765-e8fc0e68d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.344919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538ad33-9019-4e11-b765-e8fc0e68d0fc None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.345213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538ad33-9019-4e11-b765-e8fc0e68d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.345481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538ad33-9019-4e11-b765-e8fc0e68d0fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.348735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 432/848] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:26.357475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.434867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538ad33-9019-4e11-b765-e8fc0e68d0fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.436340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538ad33-9019-4e11-b765-e8fc0e68d0fc None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.451632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:26.470944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035e1e-ab68-41ab-8bf2-87b1305cb785 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.513422 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.514256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.514621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.514984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.522516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.524711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:26.536231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:26.537164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 433/849] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:26.538111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.557308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 417/850] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:36:26.558224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538ad33-9019-4e11-b765-e8fc0e68d0fc None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.578552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:26.661991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f336d-3cfc-4467-910f-066792ff8b52 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.676261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538ad33-9019-4e11-b765-e8fc0e68d0fc None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.693504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:26.703838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.713400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.713400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.713400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.713400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.713400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.716411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.717021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.717300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:26.717480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:26.718394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:26.723856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:26.724600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.724600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:26.724740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 434/851] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:26.724808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.724875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.724942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.748936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 418/852] 10.222.0.94 () {56 vars in 1135 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:26.748936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.835658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:26.846310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b8f75-3abb-45da-b4c1-847f606e20e6 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.850436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e708-17bc-4c88-a342-00148cf1a875 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:26.885490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.886328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.886624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.886869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.887264 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:26.887872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:26.888250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:26.888413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:26.889043 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:26.889345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.890711 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.899195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 419/853] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:26.899195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.903098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:26.904559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:26.922355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:26.923581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:26.925077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:26.925077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:26.925077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:26.925077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 435/854] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:26.925077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:26.940030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:27.023394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d26fe-801d-4b9c-81ef-70a236c153df None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.037737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7540cbd-fb30-480b-8855-5f4845d2664e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.060180 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879791fa-3e57-47ee-90c3-a850ac91eec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.060940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879791fa-3e57-47ee-90c3-a850ac91eec1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.061269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879791fa-3e57-47ee-90c3-a850ac91eec1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.061558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879791fa-3e57-47ee-90c3-a850ac91eec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.079357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.080210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.080666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.080986 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.081680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:27.082022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.083709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.089555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 436/855] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:27.090836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:27.107230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:27.161666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879791fa-3e57-47ee-90c3-a850ac91eec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.163379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879791fa-3e57-47ee-90c3-a850ac91eec1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.204065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:27.233782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee14f5-8459-46b1-b184-2f42447e7700 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.276308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b19ed-2e58-4e04-8326-cd568ffb237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.277261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b19ed-2e58-4e04-8326-cd568ffb237b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.277710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b19ed-2e58-4e04-8326-cd568ffb237b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.278112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b19ed-2e58-4e04-8326-cd568ffb237b None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.279032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b19ed-2e58-4e04-8326-cd568ffb237b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:27.279493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b19ed-2e58-4e04-8326-cd568ffb237b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.281045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b19ed-2e58-4e04-8326-cd568ffb237b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.283126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 420/856] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:36:27.284380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eabda-47cb-42ee-a742-c73a5d6c9635 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.284663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eabda-47cb-42ee-a742-c73a5d6c9635 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.284938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eabda-47cb-42ee-a742-c73a5d6c9635 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.286444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 421/857] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:27.289642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7eabda-47cb-42ee-a742-c73a5d6c9635 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:27.295904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 437/858] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:27.297372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45664b4f-5f6e-43b2-9e14-4a3fb2298448 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.297792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45664b4f-5f6e-43b2-9e14-4a3fb2298448 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.298216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45664b4f-5f6e-43b2-9e14-4a3fb2298448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.316649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-45664b4f-5f6e-43b2-9e14-4a3fb2298448 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:27.318075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-45664b4f-5f6e-43b2-9e14-4a3fb2298448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:27.384271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:27.393485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 438/859] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:27.394489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45664b4f-5f6e-43b2-9e14-4a3fb2298448 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:27.410199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7eabda-47cb-42ee-a742-c73a5d6c9635 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.458296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.459064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.459400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.460296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.463578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:27.476451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.478273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:27.499614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:27.500835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:27.501949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:27.501949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:27.501949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:27.502413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 422/860] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:27.507677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:27.515964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45664b4f-5f6e-43b2-9e14-4a3fb2298448 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.555447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.556543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.557025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.557490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.585640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.587845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:27.600023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:27.601049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:27.602517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 439/861] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:27.607899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:27.640854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d35a8-e766-497d-a313-04a5ed4798b0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.679216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.680450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.680674 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.680896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.681738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:27.681845 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.683241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.685692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:27.696229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 423/862] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:27.697412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:27.746370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8052fc4-f829-49d9-9963-ca1cbe7e4726 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.755629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:27.787738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.788354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.788627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.788914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.789854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:27.790145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.791630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.804610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 440/863] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:27.806316 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:27.828434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c7688-685e-4c1d-a1c1-7ae409bdf262 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.868134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.869761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.870103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.870606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.871031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.872345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:27.881902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:27.883056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:27.883911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 424/864] 10.222.0.94 () {56 vars in 1132 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:27.885903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:27.920706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2691262-cdd4-42ca-99eb-de66903de9a0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:27.941176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:27.960195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:27.960827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:27.961263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:27.961544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:27.968671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:27.970105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:27.982379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:27.983484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 441/865] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:27.984537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:28.001578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1106f-9715-478d-bf1d-cccf005db5c4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:28.012979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:28.041933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:28.043345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:28.043970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:28.044664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:28.045436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=9e1dd86804944fd1806ec289181b361e) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:28.046174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:28.047660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:28.062948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 425/866] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/role_assignments?user.id=9e1dd86804944fd1806ec289181b361e&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:28.062948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:28.114334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807b568-d228-4541-b69f-90ff2d766621 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:28.154685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:28.155456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:28.155801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:28.156194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:28.157060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:28.157455 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:28.159120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:28.165802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 442/867] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:28.172244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:28.205792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03564d-8d33-4c61-a598-476d2865e76f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:28.228847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:28.248676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:28.251953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:28.251953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:28.251953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:28.251953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:28.251953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:28.252443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:28.269394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 426/868] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:28.276211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:28.317182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cd48f-e6a4-4296-b44c-b21ffda6e0e7 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:28.319162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:28.359627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064680b-7b88-4ac6-b85c-71a4a9887771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:28.360237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064680b-7b88-4ac6-b85c-71a4a9887771 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:28.360516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064680b-7b88-4ac6-b85c-71a4a9887771 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:28.360885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064680b-7b88-4ac6-b85c-71a4a9887771 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:28.361715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064680b-7b88-4ac6-b85c-71a4a9887771 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:28.362016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064680b-7b88-4ac6-b85c-71a4a9887771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:28.363426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064680b-7b88-4ac6-b85c-71a4a9887771 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:28.369050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 443/869] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:28.408217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca619471-786a-472f-9a5e-ba5023dbee22 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:28.449503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46165908-a940-45e6-bf56-e43e0652b72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:28.450113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46165908-a940-45e6-bf56-e43e0652b72e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:28.450390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46165908-a940-45e6-bf56-e43e0652b72e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:28.450694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46165908-a940-45e6-bf56-e43e0652b72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:28.451494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46165908-a940-45e6-bf56-e43e0652b72e None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=b92a2d92e27d4fe48b42f9e04be92372) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:28.451863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46165908-a940-45e6-bf56-e43e0652b72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:28.453354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46165908-a940-45e6-bf56-e43e0652b72e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:28.467124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 427/870] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:28 2022] GET /identity/v3/role_assignments?user.id=b92a2d92e27d4fe48b42f9e04be92372&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:29.116487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b8b0e-c437-469f-9d5c-881e9054a12f None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:29.117008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b8b0e-c437-469f-9d5c-881e9054a12f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:29.117008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b8b0e-c437-469f-9d5c-881e9054a12f None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:29.119611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 444/871] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:29.124870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea910a-d095-4980-a534-43a7beaad030 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:29.125299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea910a-d095-4980-a534-43a7beaad030 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:29.125469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea910a-d095-4980-a534-43a7beaad030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:29.145701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-71ea910a-d095-4980-a534-43a7beaad030 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:29.146908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-71ea910a-d095-4980-a534-43a7beaad030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:29.223098 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 428/872] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:29.495943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b8b0e-c437-469f-9d5c-881e9054a12f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:29.608217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b8b0e-c437-469f-9d5c-881e9054a12f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:29.611154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:29.645775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe413a7-22fb-4b0d-b9b5-b3b728a09204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:29.646386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe413a7-22fb-4b0d-b9b5-b3b728a09204 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:29.646659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe413a7-22fb-4b0d-b9b5-b3b728a09204 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:29.646939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe413a7-22fb-4b0d-b9b5-b3b728a09204 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:29.647585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe413a7-22fb-4b0d-b9b5-b3b728a09204 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:29.649798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe413a7-22fb-4b0d-b9b5-b3b728a09204 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:29.724239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 445/873] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:36:29 2022] POST /identity/v3/services => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:30.182168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa45cf-39f0-4bd2-a2b2-379f1a8d083c None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.182687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa45cf-39f0-4bd2-a2b2-379f1a8d083c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.182687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa45cf-39f0-4bd2-a2b2-379f1a8d083c None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.184456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 429/874] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:30.189984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70ffee-ca49-47f7-8ff0-c8c886f68648 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.190431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70ffee-ca49-47f7-8ff0-c8c886f68648 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.190613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70ffee-ca49-47f7-8ff0-c8c886f68648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.210367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4a70ffee-ca49-47f7-8ff0-c8c886f68648 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:30.212260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4a70ffee-ca49-47f7-8ff0-c8c886f68648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:30.303700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 446/875] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:30.371527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e40d-1993-4e79-8529-1021e6caa4da None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.371840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e40d-1993-4e79-8529-1021e6caa4da None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.372070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e40d-1993-4e79-8529-1021e6caa4da None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.373713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 430/876] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:30.380785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06dfff-be2a-43c9-9ecb-1759e7bce65a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.381077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06dfff-be2a-43c9-9ecb-1759e7bce65a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.381350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06dfff-be2a-43c9-9ecb-1759e7bce65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.401661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2f06dfff-be2a-43c9-9ecb-1759e7bce65a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:30.402861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2f06dfff-be2a-43c9-9ecb-1759e7bce65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:30.477421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 447/877] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:30.480145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e00bc-bbee-4ab3-8e23-970e8b840456 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.480664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e00bc-bbee-4ab3-8e23-970e8b840456 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.481135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e00bc-bbee-4ab3-8e23-970e8b840456 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.482942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 431/878] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:30.488635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b02881-700c-420d-b191-8e0b42010b56 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.488920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b02881-700c-420d-b191-8e0b42010b56 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.489192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b02881-700c-420d-b191-8e0b42010b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.509117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b5b02881-700c-420d-b191-8e0b42010b56 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:30.510315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b5b02881-700c-420d-b191-8e0b42010b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:30.535639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:30.586057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 448/879] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:30.592525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e00bc-bbee-4ab3-8e23-970e8b840456 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:30.628345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:30.706775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e00bc-bbee-4ab3-8e23-970e8b840456 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:30.743005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddf2ef-ba22-4052-86de-d6a32d9745b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:30.743716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddf2ef-ba22-4052-86de-d6a32d9745b3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.743996 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddf2ef-ba22-4052-86de-d6a32d9745b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.744265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddf2ef-ba22-4052-86de-d6a32d9745b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.755073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddf2ef-ba22-4052-86de-d6a32d9745b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ffddf2ef-ba22-4052-86de-d6a32d9745b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:30.760956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:30.761914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:30.762748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 432/880] 10.222.0.94 () {56 vars in 1120 bytes} [Fri Aug 26 15:36:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:30.768348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b02881-700c-420d-b191-8e0b42010b56 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:30.782122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddf2ef-ba22-4052-86de-d6a32d9745b3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:30.886508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b02881-700c-420d-b191-8e0b42010b56 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:30.893120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:30.906756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddf2ef-ba22-4052-86de-d6a32d9745b3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:30.926147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:30.926754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.927129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.927504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.928451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:30.928561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:30.930087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:30.943519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 449/881] 10.222.0.94 () {56 vars in 1115 bytes} [Fri Aug 26 15:36:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:30.943519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:30.949509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b466e23-f42a-4569-a384-2717f244f3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:30.950127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b466e23-f42a-4569-a384-2717f244f3d2 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:30.950403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b466e23-f42a-4569-a384-2717f244f3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:30.950695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b466e23-f42a-4569-a384-2717f244f3d2 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:30.951447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b466e23-f42a-4569-a384-2717f244f3d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:30.952828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b466e23-f42a-4569-a384-2717f244f3d2 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:30.959770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:31.044608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96cad7-d79f-4e54-9899-5cc2ba3e0148 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:31.061141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 433/882] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:36:30 2022] POST /identity/v3/services => generated 269 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:31.062223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b466e23-f42a-4569-a384-2717f244f3d2 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:31.091496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:31.092281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.092615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.092997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.093979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:31.096385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:31.107036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:31.108110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:31.111799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 450/883] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:31.114616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:31.209363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b466e23-f42a-4569-a384-2717f244f3d2 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:31.224495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:31.259420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:31.260075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.260373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.260616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.268412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255eabfc-a7cc-4a56-ad5a-a942ac7845d7 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:31.272354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:31.274156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:31.282763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:31.289547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:31.293172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:31.294081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:31.294081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:31.294081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:31.294081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 434/884] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:31.296973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:31.314595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:31.315235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.315504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.315763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.316394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:31.316685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:31.318048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:31.326886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 451/885] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:31.332582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:31.412615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ac000-255b-4c38-a969-9173756ff273 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:31.458754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:31.459389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.459665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.459930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.460570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:31.460903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:31.462402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:31.481480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 435/886] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:31.483999 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:31.490468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a1529-6b15-4dff-9451-06f52339a56b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:31.528301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:31.529040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.529273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.529546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.530253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:31.530605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:31.532051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:31.534631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:31.537962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 452/887] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:31.581210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:31.633360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c8bc4-2410-4bd8-b76b-d4a3d2741c0c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:31.673289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee827ee3-80f3-4351-af70-1b4bf312ace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:31.673946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee827ee3-80f3-4351-af70-1b4bf312ace3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.674222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee827ee3-80f3-4351-af70-1b4bf312ace3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.674493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee827ee3-80f3-4351-af70-1b4bf312ace3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.683753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee827ee3-80f3-4351-af70-1b4bf312ace3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:31.685217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee827ee3-80f3-4351-af70-1b4bf312ace3 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ee827ee3-80f3-4351-af70-1b4bf312ace3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:31.698494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:31.699512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:31.700453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 436/888] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:36:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:31.706882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:31.768213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756415-67ca-44de-a0f7-1b3ded507d09 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.768555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756415-67ca-44de-a0f7-1b3ded507d09 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.768754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756415-67ca-44de-a0f7-1b3ded507d09 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.770371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 437/889] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:31.775680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821851-c10b-4c61-b020-3d9531e91caa None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.775960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821851-c10b-4c61-b020-3d9531e91caa None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.776228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821851-c10b-4c61-b020-3d9531e91caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.795415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e8821851-c10b-4c61-b020-3d9531e91caa None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:31.796883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e8821851-c10b-4c61-b020-3d9531e91caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:31.830411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39109c-1ba4-43ad-a1e6-055b14a7e2b4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:31.875015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:31.875331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 438/890] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:31.875443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:31.875678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:31.875943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:31.876669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:31.876965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:31.878384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:31.897547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 453/891] 10.222.0.94 () {56 vars in 1149 bytes} [Fri Aug 26 15:36:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:31.906336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8821851-c10b-4c61-b020-3d9531e91caa None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:31.964744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:32.019215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8821851-c10b-4c61-b020-3d9531e91caa None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:32.057002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b44459-360f-4d09-84a0-91cc004daada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:32.057631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b44459-360f-4d09-84a0-91cc004daada None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:32.057908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b44459-360f-4d09-84a0-91cc004daada None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:32.058176 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b44459-360f-4d09-84a0-91cc004daada None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:32.131266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b44459-360f-4d09-84a0-91cc004daada None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:32.132756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b44459-360f-4d09-84a0-91cc004daada None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:32.158704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:32.197234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:32.225514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:32.267039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 439/892] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:31 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:36:32.272226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b44459-360f-4d09-84a0-91cc004daada None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:32.286608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c612d16-4e97-47ce-b9e9-80eadc437b3c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:32.330782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:32.331829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:32.332307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:32.332797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:32.333579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:32.335176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:32.344927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:32.345902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:32.346708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:32.354721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 454/893] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:32.355840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:32.406140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b44459-360f-4d09-84a0-91cc004daada None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:32.447625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:32.448641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:32.449124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:32.449601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:32.463406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:32.465279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:32.481639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79718-63ca-4a0c-a96d-5b20714630dd None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:32.482876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:32.483985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:32.484942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:32.484942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:32.484942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:32.486759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 440/894] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:32.498975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:32.498975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:32.516107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:32.527752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:32.529882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:32.530159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:32.530425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:32.531084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:32.531385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:32.533895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:32.540478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 455/895] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:32.546808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:32.628257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066efc81-5d1c-4202-978f-6c34878f16fa None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:32.668798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:32.669465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:32.669748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:32.670066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:32.670823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:32.671154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:32.672584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:32.674021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49e87-41d6-4238-abae-e4c23d561836 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:32.687384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 441/896] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:32.698962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:32.716640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:32.717256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:32.717590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:32.717860 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:32.718545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:32.718854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:32.720260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:32.726034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 456/897] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:32.736235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:32.776661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:32.808622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:32.831843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9ccab-a757-4a15-ac53-5cb3bb29219c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:32.864639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f7321-949d-4ec4-95e3-a955d938086f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:32.878174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:32.878815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:32.879123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:32.879428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:32.888884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:32.890599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:32.902845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:32.903668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:32.904094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:32.904531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:32.904782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:32.905808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:32.906724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 442/898] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:32.908339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] RBAC: Adding query filter params (interface=public, service_id=279fcbe3ec3f4e96ba468daf452a5260, region_id=RegionOne) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:32.908738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:32.910047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:32.912504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:32.921638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 457/899] 10.222.0.94 () {56 vars in 1251 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/endpoints?service_id=279fcbe3ec3f4e96ba468daf452a5260&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:32.933018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:33.033480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6afb0-00a3-4064-837e-29e66cad5b4e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:33.059009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:33.063593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00fb68-4aac-4071-9ef5-72d5377126eb None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:33.074798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:33.081829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.081829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.081829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.081829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:33.081829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:33.081829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:33.082286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:33.093287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 443/900] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:33.099672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:33.110292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f2e77-fd7d-40be-9d37-0451121345a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:33.111445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f2e77-fd7d-40be-9d37-0451121345a0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.111727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f2e77-fd7d-40be-9d37-0451121345a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.111994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f2e77-fd7d-40be-9d37-0451121345a0 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.113214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f2e77-fd7d-40be-9d37-0451121345a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:33.115579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f2e77-fd7d-40be-9d37-0451121345a0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:33.127184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 458/901] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:36:32 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:36:33.129673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd81d-21e6-410c-9e19-ad38bcec3c3f None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.130192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd81d-21e6-410c-9e19-ad38bcec3c3f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.130603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd81d-21e6-410c-9e19-ad38bcec3c3f None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.138338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 459/902] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:33.140725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de445a8-5c87-4fc5-8720-fb0ce359f4e9 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.141757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de445a8-5c87-4fc5-8720-fb0ce359f4e9 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.142190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de445a8-5c87-4fc5-8720-fb0ce359f4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.167952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5de445a8-5c87-4fc5-8720-fb0ce359f4e9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:33.170304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5de445a8-5c87-4fc5-8720-fb0ce359f4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:33.236419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48136-93dc-42d5-a354-c5f4106a7571 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:33.263382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 460/903] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:33.280832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6a39e-8768-42ab-9aa7-ef25b5c30dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:33.281607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6a39e-8768-42ab-9aa7-ef25b5c30dfb None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.281985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6a39e-8768-42ab-9aa7-ef25b5c30dfb None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.282358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6a39e-8768-42ab-9aa7-ef25b5c30dfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.283219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6a39e-8768-42ab-9aa7-ef25b5c30dfb None admin] RBAC: Adding query filter params (role.id=d6c6c174328447a29ce8d67c9cef9397, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=9e1dd86804944fd1806ec289181b361e) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:33.283628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6a39e-8768-42ab-9aa7-ef25b5c30dfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:33.285161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6a39e-8768-42ab-9aa7-ef25b5c30dfb None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:33.292186 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 444/904] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:33 2022] GET /identity/v3/role_assignments?user.id=9e1dd86804944fd1806ec289181b361e&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=d6c6c174328447a29ce8d67c9cef9397 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:33.538480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de445a8-5c87-4fc5-8720-fb0ce359f4e9 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:33.637741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ac70-0abf-45b0-9b1b-a722159e4060 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.638072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ac70-0abf-45b0-9b1b-a722159e4060 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.638289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739ac70-0abf-45b0-9b1b-a722159e4060 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.639910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 445/905] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:33.644932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cce74-f827-49b1-9aec-1078d95a0130 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.645062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de445a8-5c87-4fc5-8720-fb0ce359f4e9 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:33.645245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cce74-f827-49b1-9aec-1078d95a0130 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.645481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cce74-f827-49b1-9aec-1078d95a0130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.659196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:33.666248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-659cce74-f827-49b1-9aec-1078d95a0130 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:33.668016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-659cce74-f827-49b1-9aec-1078d95a0130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:33.681266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:33.681899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.682169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.682432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.683044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:33.684411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:33.693332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:33.694301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:33.695770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:33.699064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 461/906] 10.222.0.94 () {56 vars in 1132 bytes} [Fri Aug 26 15:36:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:33.701213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:33.744040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:33.751033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 446/907] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:33.810517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bced9-1394-4d05-9e11-c29d1d98328f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:33.846567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:33.847208 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:33.847482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:33.847743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:33.848363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:33.848656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:33.850005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:33.860022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 462/908] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:33.863642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cce74-f827-49b1-9aec-1078d95a0130 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:33.969410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cce74-f827-49b1-9aec-1078d95a0130 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:34.006127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:34.006732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:34.007033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:34.007328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:34.008546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:34.008842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:34.010293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:34.015690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 447/909] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:34.021538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:34.043727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:34.061295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:34.090051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:34.139859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93820a2d-8b6a-4ef0-9d65-16ba42684ebe None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:34.160022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d141e9c-2be7-488d-be66-8503879ad34b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:34.177696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:34.178681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:34.179213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:34.179575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:34.180276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] RBAC: Adding query filter params (interface=public, service_id=47eb24fd24be4d25868d23539e8a690c, region_id=RegionOne) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:34.180667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:34.182185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:34.192410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 463/910] 10.222.0.94 () {56 vars in 1251 bytes} [Fri Aug 26 15:36:34 2022] GET /identity/v3/endpoints?service_id=47eb24fd24be4d25868d23539e8a690c&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:34.194832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:34.198089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5aed1-4853-4a2c-8e71-a8b2b7c8347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:34.198878 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5aed1-4853-4a2c-8e71-a8b2b7c8347b None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:34.199187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5aed1-4853-4a2c-8e71-a8b2b7c8347b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:34.200307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5aed1-4853-4a2c-8e71-a8b2b7c8347b None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:34.200974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc5aed1-4853-4a2c-8e71-a8b2b7c8347b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:34.202340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc5aed1-4853-4a2c-8e71-a8b2b7c8347b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:34.305086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 448/911] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:36:34 2022] POST /identity/v3/services => generated 241 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:36:34.311272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56e0c1-e6e9-4359-9d57-c5b067390e25 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:34.315292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:34.361442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0f4fa-66bc-410e-9e38-789486735868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:34.362059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f4fa-66bc-410e-9e38-789486735868 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:34.362337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f4fa-66bc-410e-9e38-789486735868 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:34.362606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f4fa-66bc-410e-9e38-789486735868 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:34.363265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0f4fa-66bc-410e-9e38-789486735868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:34.365175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0f4fa-66bc-410e-9e38-789486735868 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:34.379228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 464/912] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:36:34 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:36:35.192120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ca56-241b-4b43-83e4-98a9ca52d8bd None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:35.192540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ca56-241b-4b43-83e4-98a9ca52d8bd None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:35.192718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0ca56-241b-4b43-83e4-98a9ca52d8bd None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:35.194282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 449/913] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:35.200222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6c6b0-7e94-421a-a285-476378fd5cd0 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:35.200512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6c6b0-7e94-421a-a285-476378fd5cd0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:35.200784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6c6b0-7e94-421a-a285-476378fd5cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:35.219278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-f1e6c6b0-7e94-421a-a285-476378fd5cd0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:35.220481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f1e6c6b0-7e94-421a-a285-476378fd5cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:35.229394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929af8c-e8e0-433a-9501-dab840d0ff49 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:35.229713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929af8c-e8e0-433a-9501-dab840d0ff49 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:35.229939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929af8c-e8e0-433a-9501-dab840d0ff49 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:35.231555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 450/914] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:35.237263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0195c-61ce-4c83-bde1-da293ea95eda None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:35.237824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0195c-61ce-4c83-bde1-da293ea95eda None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:35.238318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0195c-61ce-4c83-bde1-da293ea95eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:35.256987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-fcb0195c-61ce-4c83-bde1-da293ea95eda None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:35.259107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-fcb0195c-61ce-4c83-bde1-da293ea95eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:35.306430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 465/915] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:35.340599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 451/916] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:35.595034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6c6b0-7e94-421a-a285-476378fd5cd0 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:35.626450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0195c-61ce-4c83-bde1-da293ea95eda None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:35.665984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:35.706269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:35.719003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6c6b0-7e94-421a-a285-476378fd5cd0 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:35.755208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0195c-61ce-4c83-bde1-da293ea95eda None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:35.764852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:35.766349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:35.766881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:35.767470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:35.768664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:35.770897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:35.787822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:35.789553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 15:36:35.790564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:35.792463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 466/917] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:35.798218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:35.805800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfa3a7-4bb5-436f-8aa1-b246713acc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:35.806778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfa3a7-4bb5-436f-8aa1-b246713acc6c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:35.807307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfa3a7-4bb5-436f-8aa1-b246713acc6c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:35.807871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfa3a7-4bb5-436f-8aa1-b246713acc6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:35.819820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfa3a7-4bb5-436f-8aa1-b246713acc6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-92dfa3a7-4bb5-436f-8aa1-b246713acc6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:35.826345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:35.827353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:35.828178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:35.830895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 452/918] 10.222.0.94 () {56 vars in 1120 bytes} [Fri Aug 26 15:36:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:35.837269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfa3a7-4bb5-436f-8aa1-b246713acc6c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:35.929172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891f50-e807-4460-bf6e-c389c12a9547 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:35.941584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:35.952136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfa3a7-4bb5-436f-8aa1-b246713acc6c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:35.970315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:35.971363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:35.971920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:35.972402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:35.973299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:35.973811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:35.975445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:35.986234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 467/919] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:35.988113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:35.990622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:35.991384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:35.991829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:35.992286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:35.992976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:35.994308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:35.995073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:36.009569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 453/920] 10.222.0.94 () {56 vars in 1115 bytes} [Fri Aug 26 15:36:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:36.010734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:36.012470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:36.106848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1fae0-ee0c-44a2-9eaf-125ed55c7320 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:36.127138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd89a72-df18-49ea-959a-5629050a91d0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:36.154229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:36.155043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.155862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.156237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.157282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:36.157689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:36.159374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:36.171616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 468/921] 10.222.0.94 () {56 vars in 1114 bytes} [Fri Aug 26 15:36:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:36.172880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:36.179138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:36.179813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.180092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.180366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.192453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:36.194469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:36.211499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:36.212557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:36.213435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:36.213435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:36.213435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:36.217779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 454/922] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:36.218675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:36.224193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:36.287413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:36.311276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f411b-2c4c-4142-ab1b-df3ba5fd9f38 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:36.344076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28879757-f02a-46b6-aad5-1217e830464d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:36.349699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b17d5-be57-42ff-97b3-ebc6fda4d634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:36.350313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b17d5-be57-42ff-97b3-ebc6fda4d634 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.350592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b17d5-be57-42ff-97b3-ebc6fda4d634 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.350866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b17d5-be57-42ff-97b3-ebc6fda4d634 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.351654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b17d5-be57-42ff-97b3-ebc6fda4d634 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:36.353179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b17d5-be57-42ff-97b3-ebc6fda4d634 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:36.378259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0afb2-719a-46ff-b012-dc471473dfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:36.378852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0afb2-719a-46ff-b012-dc471473dfeb None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.379154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0afb2-719a-46ff-b012-dc471473dfeb None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.379422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0afb2-719a-46ff-b012-dc471473dfeb None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.380050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0afb2-719a-46ff-b012-dc471473dfeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:36.380340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0afb2-719a-46ff-b012-dc471473dfeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:36.381745 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0afb2-719a-46ff-b012-dc471473dfeb None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:36.394055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 455/923] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:36.395315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec9df1-a6da-446e-90ad-04d5e7c77184 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.395589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec9df1-a6da-446e-90ad-04d5e7c77184 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.395932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec9df1-a6da-446e-90ad-04d5e7c77184 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.397173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 456/924] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:36.398461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5a582-4878-47c3-bae9-1116bc533e40 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.398595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5a582-4878-47c3-bae9-1116bc533e40 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.398771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5a582-4878-47c3-bae9-1116bc533e40 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.400220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 457/925] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:36.400958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5a582-4878-47c3-bae9-1116bc533e40 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:36.442257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 469/926] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:36:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:36.443536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06784a1a-2e93-40a2-978c-b7342aba10ba None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.443819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06784a1a-2e93-40a2-978c-b7342aba10ba None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.444088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06784a1a-2e93-40a2-978c-b7342aba10ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.464580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-06784a1a-2e93-40a2-978c-b7342aba10ba None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:36.465835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-06784a1a-2e93-40a2-978c-b7342aba10ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:36.487835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:36.544072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5a582-4878-47c3-bae9-1116bc533e40 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:36.556666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 470/927] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:36.557668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683cb8-f350-4f8e-a605-7c620501a3f3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.557961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683cb8-f350-4f8e-a605-7c620501a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.558230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683cb8-f350-4f8e-a605-7c620501a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.588571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-55683cb8-f350-4f8e-a605-7c620501a3f3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:36.589959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-55683cb8-f350-4f8e-a605-7c620501a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:36.605860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:36.606895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.607328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.607624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.615622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:36.618439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:36.635896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:36.636897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:36.637827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:36.638373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 458/928] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:36:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:36.644775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:36.673632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:36.682720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 471/929] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:36.765644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6d70d-4568-459d-9f56-d5fdc8c042d6 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:36.787753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:36.803634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:36.804309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.805086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.805430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.806486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:36.806792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:36.808262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:36.829688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 459/930] 10.222.0.94 () {56 vars in 1149 bytes} [Fri Aug 26 15:36:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:36.831191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683cb8-f350-4f8e-a605-7c620501a3f3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:36.835865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:36.950172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bff51f-966c-4ae4-8aa5-164a1b095817 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:36.955578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683cb8-f350-4f8e-a605-7c620501a3f3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:36.986410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:36.987075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:36.987346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:36.987615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:36.998994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.000449 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.005494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.006143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.006419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.006689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.007321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.010133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:37.015900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:37.016827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:37.018258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:37.018258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:37.018258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:37.018258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 460/931] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:37.018258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:37.024676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:37.025670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:37.027167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 472/932] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:37.028274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.059061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:37.095458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:37.137913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468955b6-eade-4481-9767-b233d386a3a5 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.139899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764b4bd-cdb4-4b93-8c01-6a031908cfa4 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.174686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.175336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.175609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.175879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.176524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:37.176821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.178195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.183569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.184294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.184678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.185028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.185684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.187641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.196429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 473/933] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:37.196429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:37.202616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:37.203594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 15:36:37.204523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:37.208569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 461/934] 10.222.0.94 () {56 vars in 1132 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:37.209643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.305554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e077732-55d5-4659-8a8a-81ad8f86d7ec None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.306672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:37.313576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e8257-a302-491b-af20-b8f01cbb9006 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.331927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:37.345715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.346409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.346696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.347076 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.347812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:37.348123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.349587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.349777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.350527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.350763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.351066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.355708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 474/935] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:37.356895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.362063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.363429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:37.375871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:37.376992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 462/936] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:37.383402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.477213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cda67-c063-4166-8fd5-c3eafa3ce4e5 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.502811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8a22f-4a44-4f84-86c8-6aee1819184d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.517158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.517819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.518091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.518599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.519031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:37.519329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.520681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.531042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 475/937] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:37.531042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.546723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.547384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.547630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.547902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.548530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:37.548827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.550204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.555929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 463/938] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:37.556733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.584884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:37.597692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:37.656556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d9e35-a95d-430e-aee7-6083b32a534c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.672968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98c8f7-5521-42f8-9792-64bf8f6ed93d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.698071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.698774 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.698997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.699395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.700098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:37.700401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.701832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.721432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 476/939] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:37.721432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.726006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.726634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.727031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.727422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.728016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:37.728397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.729706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.739702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 464/940] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:37.739702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.825968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:37.836185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df117b-12f9-4d14-b892-7ba70c4735a8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.844870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45bf56-3f96-4a99-b9d4-b61bda30a9d9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:37.848527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:37.873833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.874455 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.874730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.875034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.875740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] RBAC: Adding query filter params (interface=public, service_id=756613e729c9465bb713a747437c0ea5, region_id=RegionOne) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:37.876204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.877751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.883298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c41327-264d-4af5-9f66-a163cfe09a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:37.883978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c41327-264d-4af5-9f66-a163cfe09a24 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:37.884233 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c41327-264d-4af5-9f66-a163cfe09a24 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:37.884586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 477/941] 10.222.0.94 () {56 vars in 1251 bytes} [Fri Aug 26 15:36:37 2022] GET /identity/v3/endpoints?service_id=756613e729c9465bb713a747437c0ea5&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:37.885386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:37.887322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c41327-264d-4af5-9f66-a163cfe09a24 None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/9e1dd86804944fd1806ec289181b361e/roles/d6c6c174328447a29ce8d67c9cef9397` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:37.890254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c41327-264d-4af5-9f66-a163cfe09a24 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=9e1dd86804944fd1806ec289181b361e, role_id=d6c6c174328447a29ce8d67c9cef9397)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:37.892495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c41327-264d-4af5-9f66-a163cfe09a24 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:37.988853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a9c3-c3b4-44a8-a890-d6a2bdfe0fb0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:38.004713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 465/942] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:36:37 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/9e1dd86804944fd1806ec289181b361e/roles/d6c6c174328447a29ce8d67c9cef9397 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:36:38.005524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c41327-264d-4af5-9f66-a163cfe09a24 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:38.040173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6873fec-9c3b-4049-ac76-a23cb8d3ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:38.041282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6873fec-9c3b-4049-ac76-a23cb8d3ed81 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:38.041915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6873fec-9c3b-4049-ac76-a23cb8d3ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:38.042391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6873fec-9c3b-4049-ac76-a23cb8d3ed81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:38.043315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6873fec-9c3b-4049-ac76-a23cb8d3ed81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:38.045197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6873fec-9c3b-4049-ac76-a23cb8d3ed81 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:38.129194 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c41327-264d-4af5-9f66-a163cfe09a24 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:38.157399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 478/943] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:36:37 2022] POST /identity/v3/endpoints => generated 347 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:38.183169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a9ef2-0815-46e2-9e2f-1ac9a648153f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:38.183806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9ef2-0815-46e2-9e2f-1ac9a648153f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:38.184135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9ef2-0815-46e2-9e2f-1ac9a648153f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:38.184374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9ef2-0815-46e2-9e2f-1ac9a648153f None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:38.185009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a9ef2-0815-46e2-9e2f-1ac9a648153f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:38.186644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a9ef2-0815-46e2-9e2f-1ac9a648153f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:38.194992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 466/944] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:36:38 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:36:38.510424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7215c-377b-4ef5-be47-cb182e06d5aa None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:38.510704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7215c-377b-4ef5-be47-cb182e06d5aa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:38.511002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7215c-377b-4ef5-be47-cb182e06d5aa None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:38.512622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 479/945] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:38.518321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef255dd-6b77-45a6-8bf3-d568f8a71f0e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:38.518802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef255dd-6b77-45a6-8bf3-d568f8a71f0e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:38.518891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef255dd-6b77-45a6-8bf3-d568f8a71f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:38.526405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:38.540322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5ef255dd-6b77-45a6-8bf3-d568f8a71f0e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:38.541603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5ef255dd-6b77-45a6-8bf3-d568f8a71f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:38.619902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 467/946] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:38.900791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7215c-377b-4ef5-be47-cb182e06d5aa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:38.932534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:39.008402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7215c-377b-4ef5-be47-cb182e06d5aa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:39.044530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:39.045142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:39.045412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:39.045672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:39.046312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:39.047753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:39.050326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING keystone.server.flask.common [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 15:36:39.117555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.sql.core [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 15:36:39.117555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 15:36:39.117555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [parameters: {'id': '0a309925a37849efb351a3925f8f6b77', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 15:36:39.117555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88508) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a309925a37849efb351a3925f8f6b77.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a309925a37849efb351a3925f8f6b77. Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:36:39.135032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result.read() Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 26 15:36:39.135890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application yield resource Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise self.value Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application session.commit() Aug 26 15:36:39.136803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.session.flush() Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application compat.raise_( Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 15:36:39.137662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 15:36:39.140441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result.read() Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:36:39.142284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [parameters: {'id': '0a309925a37849efb351a3925f8f6b77', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:39.144344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:39.145582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0a309925a37849efb351a3925f8f6b77. Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:39.146681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 480/947] 10.222.0.94 () {60 vars in 1140 bytes} [Fri Aug 26 15:36:38 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:39.147378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef255dd-6b77-45a6-8bf3-d568f8a71f0e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:39.256588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef255dd-6b77-45a6-8bf3-d568f8a71f0e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:39.289579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:39.295339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c8206-723e-4f9b-92fe-a803b07163c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:39.296223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8206-723e-4f9b-92fe-a803b07163c2 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:39.296697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8206-723e-4f9b-92fe-a803b07163c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:39.297177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c8206-723e-4f9b-92fe-a803b07163c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:39.304644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c8206-723e-4f9b-92fe-a803b07163c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:39.306451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c8206-723e-4f9b-92fe-a803b07163c2 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-a29c8206-723e-4f9b-92fe-a803b07163c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:39.318598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:39.319513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:39.320437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:39.320437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:39.320437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:39.320437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:39.320437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:39.320437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:39.320437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:39.320437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:39.322757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 468/948] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:39.328918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:39.443438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:39.451104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7fdd9-7856-4122-909e-b63c96c7d10a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:39.489198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33667644-14f3-48b9-aec4-f8a5e348f585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:39.490472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667644-14f3-48b9-aec4-f8a5e348f585 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:39.490946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667644-14f3-48b9-aec4-f8a5e348f585 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:39.491245 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667644-14f3-48b9-aec4-f8a5e348f585 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:39.492121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667644-14f3-48b9-aec4-f8a5e348f585 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:39.492488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667644-14f3-48b9-aec4-f8a5e348f585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:39.495040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667644-14f3-48b9-aec4-f8a5e348f585 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:39.520652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 481/949] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:40.040127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a794c-3289-42f9-9899-e80305699bdc None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.041116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a794c-3289-42f9-9899-e80305699bdc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.041580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a794c-3289-42f9-9899-e80305699bdc None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.043175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 469/950] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:40.048845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40350e-1d67-4ee7-9ffa-3b2316b6227e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.049752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40350e-1d67-4ee7-9ffa-3b2316b6227e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.050236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40350e-1d67-4ee7-9ffa-3b2316b6227e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.070816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5b40350e-1d67-4ee7-9ffa-3b2316b6227e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:40.072838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5b40350e-1d67-4ee7-9ffa-3b2316b6227e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:40.152198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 482/951] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:40.219041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e102e5-2ecd-472c-9661-50783c634dc3 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.219817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e102e5-2ecd-472c-9661-50783c634dc3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.220284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e102e5-2ecd-472c-9661-50783c634dc3 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.222075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 470/952] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:40.228164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe45dfa-9ce7-420a-a8d6-bde00d0b260d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.229682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c550d6-f778-4aab-a406-550969cc9c85 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.229967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c550d6-f778-4aab-a406-550969cc9c85 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.230307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c550d6-f778-4aab-a406-550969cc9c85 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.230725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe45dfa-9ce7-420a-a8d6-bde00d0b260d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.231262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe45dfa-9ce7-420a-a8d6-bde00d0b260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.231908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 471/953] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:40.237115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54a10-3032-408a-b976-f808cd727d15 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.237395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54a10-3032-408a-b976-f808cd727d15 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.237676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54a10-3032-408a-b976-f808cd727d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.251869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-dbe45dfa-9ce7-420a-a8d6-bde00d0b260d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:40.253403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-dbe45dfa-9ce7-420a-a8d6-bde00d0b260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:40.257225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-fbb54a10-3032-408a-b976-f808cd727d15 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:40.258432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-fbb54a10-3032-408a-b976-f808cd727d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:40.348881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 483/954] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:40.351626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 472/955] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:40.436859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe45dfa-9ce7-420a-a8d6-bde00d0b260d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:40.446622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:40.551401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe45dfa-9ce7-420a-a8d6-bde00d0b260d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:40.588401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4887-2e82-4bbe-bc0a-af0e0b9bed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:40.589050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4887-2e82-4bbe-bc0a-af0e0b9bed27 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.589320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4887-2e82-4bbe-bc0a-af0e0b9bed27 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.589583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e4887-2e82-4bbe-bc0a-af0e0b9bed27 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.602066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e4887-2e82-4bbe-bc0a-af0e0b9bed27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-931e4887-2e82-4bbe-bc0a-af0e0b9bed27 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:40.606654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:40.607730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:40.608665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 484/956] 10.222.0.94 () {56 vars in 1120 bytes} [Fri Aug 26 15:36:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:40.616005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54a10-3032-408a-b976-f808cd727d15 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:40.632031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4887-2e82-4bbe-bc0a-af0e0b9bed27 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:40.711069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:40.727689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54a10-3032-408a-b976-f808cd727d15 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:40.753705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e4887-2e82-4bbe-bc0a-af0e0b9bed27 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:40.754064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:40.765362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:40.766041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.766273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.766541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.767228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:40.767522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:40.768877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:40.777653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 473/957] 10.222.0.94 () {56 vars in 1115 bytes} [Fri Aug 26 15:36:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:40.782305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:40.800854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:40.801491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.801769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.802039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.802632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:40.804037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:40.813293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:40.814363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:40.815325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:40.819260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 485/958] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:40.820341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:40.892341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfac699-e000-4361-9db2-1f4cd43d7add None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:40.933271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae473add-f09e-49bf-8e86-29af5c6fd998 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:40.935766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:40.936402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.936677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.936948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.937627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:40.939220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:40.948914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:40.949911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:40.950727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 474/959] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:40.951581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:40.972965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:40.980788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:40.981424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:40.981694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:40.981952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:40.993289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.040603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.040783 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:41.041211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:41.042252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:41.043260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:41.043260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:41.043260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:41.043260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 486/960] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:41.043260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:41.074616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ff88a-d2ea-448a-85ca-0089bc19e1f9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.111093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.111704 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.111978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.112244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.112865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:41.113162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.114507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.119949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 475/961] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:41.120580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:41.131044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a337ab6-4467-4674-b8a7-ce50f8b3a339 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.167682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.168210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.168482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.168744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.169385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] RBAC: Adding query filter params (name=compute) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:41.169665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.171059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.176037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 487/962] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:41.176818 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:41.235671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f30a5a-7f23-437b-af05-7c3ec4a0042f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.237357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:41.252081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:41.276611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.277296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.277576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.277852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.278602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:41.278953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.280282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.293949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 476/963] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:41.294828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:41.300221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedbb75-3faf-4b28-a1cf-3df2507d205e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.340181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.340867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.341131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.341393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.342031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:41.342335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.343725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.353460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 488/964] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:41.353460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:41.408287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367489c-03c0-4f02-b3c8-a87db8816462 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.448982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.449424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.449705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.449976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.450612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:41.450911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.455377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.458461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 477/965] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:41.459520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:41.469672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa719f0-8cbe-4f5f-af65-92565a46bc5f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.491904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:41.506852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:41.521672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.522335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.522616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.523400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.535492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.537322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:41.556408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:41.557450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 15:36:41.558401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:41.562101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 489/966] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:36:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:41.572367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:41.605008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4ee50-6b7e-47c6-b134-5fe9dfe605ed None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.649306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdae291-7780-42a8-9f8e-2b5acd8a9887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.650252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdae291-7780-42a8-9f8e-2b5acd8a9887 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.650742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdae291-7780-42a8-9f8e-2b5acd8a9887 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.651213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdae291-7780-42a8-9f8e-2b5acd8a9887 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.652076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdae291-7780-42a8-9f8e-2b5acd8a9887 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.653644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdae291-7780-42a8-9f8e-2b5acd8a9887 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.691692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6d40f-1d1c-4292-aefb-f49971973cfa None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.727671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849675f-6627-4ddb-8489-45d230732818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.728200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849675f-6627-4ddb-8489-45d230732818 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.728475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849675f-6627-4ddb-8489-45d230732818 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.728752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849675f-6627-4ddb-8489-45d230732818 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.729482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8849675f-6627-4ddb-8489-45d230732818 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:41.729783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8849675f-6627-4ddb-8489-45d230732818 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:41.731345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8849675f-6627-4ddb-8489-45d230732818 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:41.758542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 490/967] 10.222.0.94 () {56 vars in 1149 bytes} [Fri Aug 26 15:36:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:41.758542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99c4c-a90c-4bb0-8ded-246ffd18d6be None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.758542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99c4c-a90c-4bb0-8ded-246ffd18d6be None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.758542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99c4c-a90c-4bb0-8ded-246ffd18d6be None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.758542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 491/968] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:41.758542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f99c4c-a90c-4bb0-8ded-246ffd18d6be None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:41.759625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 478/969] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:36:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:41.761353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec822d9-1957-4f70-8334-d9c14950e7f7 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.761639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec822d9-1957-4f70-8334-d9c14950e7f7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.761908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec822d9-1957-4f70-8334-d9c14950e7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:41.792504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:41.807799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-bec822d9-1957-4f70-8334-d9c14950e7f7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:41.809327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bec822d9-1957-4f70-8334-d9c14950e7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:41.908916 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f99c4c-a90c-4bb0-8ded-246ffd18d6be None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:41.913600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 479/970] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:41.948710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f888603-fd00-4ddc-8413-ff901427c1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:41.949466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f888603-fd00-4ddc-8413-ff901427c1f4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:41.949843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f888603-fd00-4ddc-8413-ff901427c1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:41.950207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f888603-fd00-4ddc-8413-ff901427c1f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:42.026066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f888603-fd00-4ddc-8413-ff901427c1f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:42.027695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f888603-fd00-4ddc-8413-ff901427c1f4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:42.052884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:42.134022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 492/971] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:41 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:36:42.139346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec822d9-1957-4f70-8334-d9c14950e7f7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:42.171231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:42.200258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f888603-fd00-4ddc-8413-ff901427c1f4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:42.252391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec822d9-1957-4f70-8334-d9c14950e7f7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:42.288645 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:42.289716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:42.290178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:42.290452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:42.301657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:42.303579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:42.312814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f888603-fd00-4ddc-8413-ff901427c1f4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:42.326119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:42.327123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:42.328016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:42.328016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:42.328016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:42.331099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 480/972] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:42.334067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:42.346190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:42.357956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:42.358573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:42.358846 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:42.359164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:42.359924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:42.361409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:42.415276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.sql.core [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 15:36:42.415276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 15:36:42.415276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [parameters: {'user_id': 'fa7c12f3462e4c4dade789d4fcc77020', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 15:36:42.415276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88508) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result.read() Aug 26 15:36:42.422536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:36:42.431600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application yield resource Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application next(self.gen) Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise self.value Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application session.commit() Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 15:36:42.432571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.session.flush() Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application compat.raise_( Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 15:36:42.433513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 15:36:42.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application conn.query(q) Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application result.read() Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 15:36:42.436486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fa7c12f3462e4c4dade789d4fcc77020', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:42.438506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 26 15:36:42.441130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 26 15:36:42.449089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 26 15:36:42.449089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 26 15:36:42.449089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 15:36:42.449089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 15:36:42.449089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 15:36:42.449089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:42.449089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 493/973] 10.222.0.94 () {60 vars in 1131 bytes} [Fri Aug 26 15:36:42 2022] POST /identity/v3/users => generated 160 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:42.450527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:42.450677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:42.479963 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d1810-4e41-41a6-aade-5eeba0817571 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:42.537709 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:42.538362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:42.539259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:42.539472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:42.540131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:42.540612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:42.542160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:42.561510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 481/974] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:42.565151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:42.592433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9abe8-6f14-43fb-a59d-a6462fd42906 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:42.636896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:42.637534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:42.637815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:42.638124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:42.648400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:42.649978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:42.661930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:42.662971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:42.664469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 494/975] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:42.669404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:42.703998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f1cd4-386e-4c2f-80a8-65e8f9bd9b4d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:42.741866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:42.742535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:42.742749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:42.742964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:42.743220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:42.745595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:42.752442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:42.753928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:42.769777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:42.770699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:42.771621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:42.771621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:42.771621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:42.771621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:42.771621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:42.771621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:42.771621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:42.771621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:42.778502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 482/976] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:42.779877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:42.806179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629676c1-6b26-45b5-a1b2-0a268678de1d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:42.843377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:42.843993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:42.844266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:42.844533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:42.845423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:42.845769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:42.847587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:42.896337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 495/977] 10.222.0.94 () {56 vars in 1101 bytes} [Fri Aug 26 15:36:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:42.915229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b327-92eb-43f9-a5b8-8c44f74988e0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:42.957718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c59dbb-ede4-4d28-85a0-2efc41fef4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:42.958863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c59dbb-ede4-4d28-85a0-2efc41fef4cf None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:42.959492 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c59dbb-ede4-4d28-85a0-2efc41fef4cf None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:42.960041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c59dbb-ede4-4d28-85a0-2efc41fef4cf None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:42.961186 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c59dbb-ede4-4d28-85a0-2efc41fef4cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:42.961737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c59dbb-ede4-4d28-85a0-2efc41fef4cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:42.963530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c59dbb-ede4-4d28-85a0-2efc41fef4cf None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:42.978620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 483/978] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:42.985210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:43.102363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c50c3-a8e8-4be6-a7e5-b4d03e791433 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:43.106181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:43.141793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c746528-26da-447d-8392-ddb7965e8d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:43.143269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746528-26da-447d-8392-ddb7965e8d89 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:43.143774 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746528-26da-447d-8392-ddb7965e8d89 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:43.144418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746528-26da-447d-8392-ddb7965e8d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:43.145363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c746528-26da-447d-8392-ddb7965e8d89 None admin] RBAC: Adding query filter params (role.id=d6c6c174328447a29ce8d67c9cef9397, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=9e1dd86804944fd1806ec289181b361e) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:43.146000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c746528-26da-447d-8392-ddb7965e8d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:43.148108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c746528-26da-447d-8392-ddb7965e8d89 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:43.155371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 496/979] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:42 2022] GET /identity/v3/role_assignments?user.id=9e1dd86804944fd1806ec289181b361e&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=d6c6c174328447a29ce8d67c9cef9397 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:43.456053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bcd24-a9e4-4471-92bc-af361c181b31 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:43.456577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bcd24-a9e4-4471-92bc-af361c181b31 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:43.456577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bcd24-a9e4-4471-92bc-af361c181b31 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:43.458056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 484/980] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:43.463625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390043b-ff23-426e-bc20-109f84402133 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:43.463910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390043b-ff23-426e-bc20-109f84402133 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:43.464183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390043b-ff23-426e-bc20-109f84402133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:43.483995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d390043b-ff23-426e-bc20-109f84402133 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:43.485576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d390043b-ff23-426e-bc20-109f84402133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:43.558681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 497/981] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:43.819473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2f704-191c-454f-88dc-ca8db1fc4d73 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:43.819879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2f704-191c-454f-88dc-ca8db1fc4d73 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:43.820040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2f704-191c-454f-88dc-ca8db1fc4d73 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:43.821462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 485/982] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:43.826618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f923058-4930-4c48-92dc-545fc8298524 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:43.826904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f923058-4930-4c48-92dc-545fc8298524 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:43.827200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f923058-4930-4c48-92dc-545fc8298524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:43.833945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2f704-191c-454f-88dc-ca8db1fc4d73 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:43.846207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8f923058-4930-4c48-92dc-545fc8298524 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:43.847446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8f923058-4930-4c48-92dc-545fc8298524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:43.893468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:43.918786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 498/983] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:43.943741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2f704-191c-454f-88dc-ca8db1fc4d73 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:43.980177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401022-dfbc-43ff-9938-b207426e0339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:43.980777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401022-dfbc-43ff-9938-b207426e0339 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:43.981047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401022-dfbc-43ff-9938-b207426e0339 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:43.981308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401022-dfbc-43ff-9938-b207426e0339 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:43.981947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c401022-dfbc-43ff-9938-b207426e0339 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:43.983363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c401022-dfbc-43ff-9938-b207426e0339 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:44.152801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 486/984] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:36:43 2022] POST /identity/v3/services => generated 242 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:44.186215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f923058-4930-4c48-92dc-545fc8298524 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:44.218889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:44.304615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f923058-4930-4c48-92dc-545fc8298524 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:44.350064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:44.350707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:44.351564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:44.351879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:44.352472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:44.353837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:44.365047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:44.366209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:44.367172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 499/985] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:44.373406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401022-dfbc-43ff-9938-b207426e0339 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:44.479038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401022-dfbc-43ff-9938-b207426e0339 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:44.494691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:44.515813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:44.516434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:44.516714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:44.516984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:44.517616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:44.517915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:44.519320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:44.528973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 487/986] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:44.531990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:44.636887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:44.649554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ab56c-fd22-4f6d-b74d-fbb19231e213 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:44.685133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:44.686046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:44.686521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:44.686983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:44.687787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:44.688269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:44.689815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:44.696863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 500/987] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:44.702323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:44.808390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df63d99-9992-4d8b-a6c3-a7ef569fb159 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:44.843875 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56928519-fa0d-4aff-b7c5-832bce3af1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:44.844494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56928519-fa0d-4aff-b7c5-832bce3af1a6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:44.844771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56928519-fa0d-4aff-b7c5-832bce3af1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:44.845042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56928519-fa0d-4aff-b7c5-832bce3af1a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:44.845755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56928519-fa0d-4aff-b7c5-832bce3af1a6 None admin] RBAC: Adding query filter params (interface=admin, service_id=756613e729c9465bb713a747437c0ea5, region_id=RegionOne) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:44.846057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56928519-fa0d-4aff-b7c5-832bce3af1a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:44.847477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56928519-fa0d-4aff-b7c5-832bce3af1a6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:44.857654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 488/988] 10.222.0.94 () {56 vars in 1249 bytes} [Fri Aug 26 15:36:44 2022] GET /identity/v3/endpoints?service_id=756613e729c9465bb713a747437c0ea5&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:44.859298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:44.969386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6c622-77da-44f9-a0a7-e64362dcb67e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:45.003999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b42c7-b117-484b-98cb-0b214eb0122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:45.004597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b42c7-b117-484b-98cb-0b214eb0122b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:45.004867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b42c7-b117-484b-98cb-0b214eb0122b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:45.005130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b42c7-b117-484b-98cb-0b214eb0122b None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:45.005728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b42c7-b117-484b-98cb-0b214eb0122b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:45.007690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b42c7-b117-484b-98cb-0b214eb0122b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:45.019042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 501/989] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:36:44 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:36:45.173461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f8994-6251-413c-93ee-3c5d201038a6 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:45.173740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f8994-6251-413c-93ee-3c5d201038a6 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:45.174023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f8994-6251-413c-93ee-3c5d201038a6 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:45.175384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 489/990] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:45.181599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815717a-0c98-4631-b699-8a4c14922e5f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:45.181888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815717a-0c98-4631-b699-8a4c14922e5f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:45.182161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815717a-0c98-4631-b699-8a4c14922e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:45.201612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-9815717a-0c98-4631-b699-8a4c14922e5f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:45.203619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9815717a-0c98-4631-b699-8a4c14922e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:45.215647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:45.274954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 502/991] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:45.564871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f8994-6251-413c-93ee-3c5d201038a6 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:45.614981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:45.675991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f8994-6251-413c-93ee-3c5d201038a6 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:45.713105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:45.713810 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:45.714024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:45.714279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] PATH_INFO: `/v3/services/image` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:45.714857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:45.716239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:45.725446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:45.731369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:45.732226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 490/992] 10.222.0.94 () {56 vars in 1105 bytes} [Fri Aug 26 15:36:45 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:45.734226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815717a-0c98-4631-b699-8a4c14922e5f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:45.837631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815717a-0c98-4631-b699-8a4c14922e5f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:45.872006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315ce96-5a81-4d30-8ecd-f82cae0190a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:45.872608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315ce96-5a81-4d30-8ecd-f82cae0190a6 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:45.872881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315ce96-5a81-4d30-8ecd-f82cae0190a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:45.873148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315ce96-5a81-4d30-8ecd-f82cae0190a6 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:45.873762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315ce96-5a81-4d30-8ecd-f82cae0190a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:45.874053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315ce96-5a81-4d30-8ecd-f82cae0190a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:45.875451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315ce96-5a81-4d30-8ecd-f82cae0190a6 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:45.886250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 503/993] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:45.889526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:45.999075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516219fa-c9f0-441d-bea1-28e9c0597383 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:46.007770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:46.037933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95124b73-4932-4f5b-8544-a4f8a336e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:46.039204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124b73-4932-4f5b-8544-a4f8a336e2bc None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:46.039432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124b73-4932-4f5b-8544-a4f8a336e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:46.039666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124b73-4932-4f5b-8544-a4f8a336e2bc None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:46.040294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95124b73-4932-4f5b-8544-a4f8a336e2bc None admin] RBAC: Adding query filter params (type=image) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:46.040585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95124b73-4932-4f5b-8544-a4f8a336e2bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:46.041970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95124b73-4932-4f5b-8544-a4f8a336e2bc None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:46.047112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 491/994] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:46.149442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042171-038a-4f2f-99de-6d1cd39e86ed None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:46.149807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042171-038a-4f2f-99de-6d1cd39e86ed None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:46.149979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042171-038a-4f2f-99de-6d1cd39e86ed None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:46.151691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 504/995] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:46.157319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3aad6-8925-4423-8bd3-097fb2727540 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:46.157627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3aad6-8925-4423-8bd3-097fb2727540 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:46.157878 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3aad6-8925-4423-8bd3-097fb2727540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:46.176924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-dee3aad6-8925-4423-8bd3-097fb2727540 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:46.178145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-dee3aad6-8925-4423-8bd3-097fb2727540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:46.252132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 492/996] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:46.529862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9042171-038a-4f2f-99de-6d1cd39e86ed None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:46.543034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:46.657732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9042171-038a-4f2f-99de-6d1cd39e86ed None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:46.701054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:46.701944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:46.702392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:46.702814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:46.703595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:46.705336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:46.715081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:46.716358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:46.717380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:46.719484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 505/997] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:46.725510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3aad6-8925-4423-8bd3-097fb2727540 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:46.825117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:46.828065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3aad6-8925-4423-8bd3-097fb2727540 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:46.862279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ceb1ca-97b4-4232-9fbb-1ff98bc30fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:46.862952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb1ca-97b4-4232-9fbb-1ff98bc30fc6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:46.863854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb1ca-97b4-4232-9fbb-1ff98bc30fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:46.864141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ceb1ca-97b4-4232-9fbb-1ff98bc30fc6 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:46.865058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ceb1ca-97b4-4232-9fbb-1ff98bc30fc6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:46.865358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ceb1ca-97b4-4232-9fbb-1ff98bc30fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:46.866789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ceb1ca-97b4-4232-9fbb-1ff98bc30fc6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:46.876531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 493/998] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:46.877499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:46.957726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:46.983838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75c808-db31-4620-801e-fc510db5e6d3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:47.017512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b91893-e10c-4250-b304-e8266bfd07fe None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.017799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b91893-e10c-4250-b304-e8266bfd07fe None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.018074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b91893-e10c-4250-b304-e8266bfd07fe None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.019643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 494/999] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:47.025479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:47.025915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca664fff-1436-4244-83eb-3e77292bf01d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.026128 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.026234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca664fff-1436-4244-83eb-3e77292bf01d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.026423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.026533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca664fff-1436-4244-83eb-3e77292bf01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.026670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.027280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:47.027575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:47.028934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:47.038985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 506/1000] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:47.041323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:47.046560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ca664fff-1436-4244-83eb-3e77292bf01d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:47.047877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ca664fff-1436-4244-83eb-3e77292bf01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:47.128224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 495/1001] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:47.159583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83814-215e-406d-b9c5-d2ad1a4fcfe0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:47.196073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:47.196676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.196952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.197220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.197911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] RBAC: Adding query filter params (interface=public, service_id=def805cea2cc4b6fb3207d1b98b7b55e, region_id=RegionOne) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:47.198214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:47.199607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:47.204983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 507/1002] 10.222.0.94 () {56 vars in 1251 bytes} [Fri Aug 26 15:36:47 2022] GET /identity/v3/endpoints?service_id=def805cea2cc4b6fb3207d1b98b7b55e&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:47.210527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca664fff-1436-4244-83eb-3e77292bf01d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:47.308948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:47.316145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca664fff-1436-4244-83eb-3e77292bf01d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:47.351226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab06efe-262a-4d1f-b467-17fd90a82d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:47.352245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab06efe-262a-4d1f-b467-17fd90a82d6b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.352527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab06efe-262a-4d1f-b467-17fd90a82d6b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.352795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab06efe-262a-4d1f-b467-17fd90a82d6b None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.353396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab06efe-262a-4d1f-b467-17fd90a82d6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:47.354754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab06efe-262a-4d1f-b467-17fd90a82d6b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:47.361314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 496/1003] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:36:47 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:36:47.407363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:47.410322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:47.528260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e53bc-71fc-49e3-b1d2-32f0b1c98481 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:47.568012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24af63-9202-4342-b504-080289955510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:47.568817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24af63-9202-4342-b504-080289955510 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.568817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24af63-9202-4342-b504-080289955510 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.569127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24af63-9202-4342-b504-080289955510 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.569697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24af63-9202-4342-b504-080289955510 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:47.571275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24af63-9202-4342-b504-080289955510 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8b24af63-9202-4342-b504-080289955510 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:47.585386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:47.586663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 15:36:47.587616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:47.592019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 508/1004] 10.222.0.94 () {56 vars in 1126 bytes} [Fri Aug 26 15:36:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:47.597764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab06efe-262a-4d1f-b467-17fd90a82d6b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:47.714546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab06efe-262a-4d1f-b467-17fd90a82d6b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:47.753732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:47.754737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.755038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.755318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.756005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:47.756300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:47.757679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:47.768181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 497/1005] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:47.769120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24af63-9202-4342-b504-080289955510 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:47.844788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:47.881855 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24af63-9202-4342-b504-080289955510 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:47.918244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73dea1f-4838-4c87-a57d-c6cc613e64e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:47.918857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dea1f-4838-4c87-a57d-c6cc613e64e4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.919163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dea1f-4838-4c87-a57d-c6cc613e64e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.919429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dea1f-4838-4c87-a57d-c6cc613e64e4 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.920057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dea1f-4838-4c87-a57d-c6cc613e64e4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:47.920355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dea1f-4838-4c87-a57d-c6cc613e64e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:47.921715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dea1f-4838-4c87-a57d-c6cc613e64e4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:47.926621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 509/1006] 10.222.0.94 () {56 vars in 1122 bytes} [Fri Aug 26 15:36:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:47.933122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:47.958204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:47.997021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f7fe1-36ad-4832-87c8-be1ff4dddd47 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:47.997317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f7fe1-36ad-4832-87c8-be1ff4dddd47 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:47.997484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f7fe1-36ad-4832-87c8-be1ff4dddd47 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:47.998966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 510/1007] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:48.003963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e1ace-f958-469b-a9fc-d73291084d1e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:48.004236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e1ace-f958-469b-a9fc-d73291084d1e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:48.004503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e1ace-f958-469b-a9fc-d73291084d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:48.023121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6f2e1ace-f958-469b-a9fc-d73291084d1e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:48.024380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6f2e1ace-f958-469b-a9fc-d73291084d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:48.041410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a00b-5f67-4618-8318-376ce150047b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:48.082170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ef776-7398-48ea-ad94-21336f009aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:48.082772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef776-7398-48ea-ad94-21336f009aa9 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:48.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef776-7398-48ea-ad94-21336f009aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:48.083338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef776-7398-48ea-ad94-21336f009aa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:48.083983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ef776-7398-48ea-ad94-21336f009aa9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:48.086141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ef776-7398-48ea-ad94-21336f009aa9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:48.096251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 511/1008] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:48.175326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 498/1009] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:36:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:36:48.361521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e1ace-f958-469b-a9fc-d73291084d1e None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:48.426733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:48.468964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e1ace-f958-469b-a9fc-d73291084d1e None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:48.520078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ace494-56fd-4175-9e12-e8e41558ab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:48.520690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ace494-56fd-4175-9e12-e8e41558ab21 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:48.520963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ace494-56fd-4175-9e12-e8e41558ab21 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:48.521228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ace494-56fd-4175-9e12-e8e41558ab21 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:48.521859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ace494-56fd-4175-9e12-e8e41558ab21 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:48.523349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ace494-56fd-4175-9e12-e8e41558ab21 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:48.634607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 512/1010] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:36:48 2022] POST /identity/v3/services => generated 242 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:49.344104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6fb1e-6b9a-4ee4-a808-cb87fe934962 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:49.344673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6fb1e-6b9a-4ee4-a808-cb87fe934962 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:49.344673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6fb1e-6b9a-4ee4-a808-cb87fe934962 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:49.346176 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 499/1011] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:49.352182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d217c3-190d-4374-941b-e224c9a0fc5e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:49.352457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d217c3-190d-4374-941b-e224c9a0fc5e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:49.352734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d217c3-190d-4374-941b-e224c9a0fc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:49.371479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-86d217c3-190d-4374-941b-e224c9a0fc5e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:49.372675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-86d217c3-190d-4374-941b-e224c9a0fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:49.450308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 513/1012] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:49 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:49.724946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6fb1e-6b9a-4ee4-a808-cb87fe934962 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:49.814936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:49.835748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6fb1e-6b9a-4ee4-a808-cb87fe934962 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:49.871145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:49.871771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:49.872049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:49.872318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:49.872901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:49.874259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:49.884568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:49.885541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 15:36:49.886417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:49.888019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 500/1013] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:49.891584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d217c3-190d-4374-941b-e224c9a0fc5e None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:49.937066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:49.996863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d217c3-190d-4374-941b-e224c9a0fc5e None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:50.038004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a4368-85b1-4f2c-88e1-18f014a7a0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:50.038813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4368-85b1-4f2c-88e1-18f014a7a0dc None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.039114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4368-85b1-4f2c-88e1-18f014a7a0dc None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.039389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4368-85b1-4f2c-88e1-18f014a7a0dc None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.040029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a4368-85b1-4f2c-88e1-18f014a7a0dc None admin] RBAC: Adding query filter params (name=compute) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:50.040344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a4368-85b1-4f2c-88e1-18f014a7a0dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:50.041755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a4368-85b1-4f2c-88e1-18f014a7a0dc None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:50.052604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 514/1014] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:50.054734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:50.096491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2bc4f-3437-42f2-bbe8-8e5f505d239c None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.096716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2bc4f-3437-42f2-bbe8-8e5f505d239c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.096965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2bc4f-3437-42f2-bbe8-8e5f505d239c None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.098628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 515/1015] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:50.103801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077bd84-657b-4f1b-b260-67cd047a4b1b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.104071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077bd84-657b-4f1b-b260-67cd047a4b1b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.104342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077bd84-657b-4f1b-b260-67cd047a4b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.123284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a077bd84-657b-4f1b-b260-67cd047a4b1b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:50.124469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a077bd84-657b-4f1b-b260-67cd047a4b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:50.172350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da8a0e-54b0-4e07-9c49-c9299be87a17 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:50.198731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 516/1016] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:50 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:50.209601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:50.210903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.211221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.211480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.212109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:50.212401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:50.213754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:50.218991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 501/1017] 10.222.0.94 () {56 vars in 1112 bytes} [Fri Aug 26 15:36:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:50.224591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077bd84-657b-4f1b-b260-67cd047a4b1b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:50.262622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:50.329485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077bd84-657b-4f1b-b260-67cd047a4b1b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:50.365634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef8e85-9a32-4f05-95dd-15fcbb705ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:50.367107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8e85-9a32-4f05-95dd-15fcbb705ac7 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.367515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8e85-9a32-4f05-95dd-15fcbb705ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.367904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8e85-9a32-4f05-95dd-15fcbb705ac7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.368623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef8e85-9a32-4f05-95dd-15fcbb705ac7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:50.370007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef8e85-9a32-4f05-95dd-15fcbb705ac7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:50.466967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 517/1018] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:36:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:36:50.467170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:50.479937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:50.583950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889ff0-4ce6-4243-bc96-0246a2974062 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:50.614659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc19069-e097-492f-a003-e1cecd466d85 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.614998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc19069-e097-492f-a003-e1cecd466d85 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.615265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc19069-e097-492f-a003-e1cecd466d85 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.617242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 518/1019] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:50.623405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a47d9-30dd-44b0-b29d-5de54291f092 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.623405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a47d9-30dd-44b0-b29d-5de54291f092 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.623656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a47d9-30dd-44b0-b29d-5de54291f092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.631325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f41da0-adba-4231-b191-2cf1dec1134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:50.632033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f41da0-adba-4231-b191-2cf1dec1134f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.632313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f41da0-adba-4231-b191-2cf1dec1134f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.632581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f41da0-adba-4231-b191-2cf1dec1134f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.633919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f41da0-adba-4231-b191-2cf1dec1134f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:50.635417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f41da0-adba-4231-b191-2cf1dec1134f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:50.637031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 502/1020] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:50.642642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f41da0-adba-4231-b191-2cf1dec1134f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:50.649704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d71a47d9-30dd-44b0-b29d-5de54291f092 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:50.650955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d71a47d9-30dd-44b0-b29d-5de54291f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:50.737713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:50.739044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 519/1021] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:50.760399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f41da0-adba-4231-b191-2cf1dec1134f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:50.796129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b7da1-1f46-409a-8b24-923ae7940652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:50.796736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b7da1-1f46-409a-8b24-923ae7940652 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:50.797008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b7da1-1f46-409a-8b24-923ae7940652 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:50.797280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b7da1-1f46-409a-8b24-923ae7940652 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:50.797917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b7da1-1f46-409a-8b24-923ae7940652 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:50.799375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b7da1-1f46-409a-8b24-923ae7940652 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:50.879193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 503/1022] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:36:50 2022] POST /identity/v3/projects => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:36:51.018045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a47d9-30dd-44b0-b29d-5de54291f092 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:51.044601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:51.130939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a47d9-30dd-44b0-b29d-5de54291f092 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:51.167521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:51.168165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:51.168440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:51.168710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] PATH_INFO: `/v3/services/image` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:51.169298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:51.170663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:51.180321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:51.181353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:51.182267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 520/1023] 10.222.0.94 () {56 vars in 1105 bytes} [Fri Aug 26 15:36:51 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:51.188832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b7da1-1f46-409a-8b24-923ae7940652 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:51.294029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b7da1-1f46-409a-8b24-923ae7940652 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:51.321552 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:51.330950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:51.331565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:51.333997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:51.333997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:51.333997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] RBAC: Adding query filter params (name=image) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:51.333997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:51.334460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:51.339585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 504/1024] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:51.345145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:51.440375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:51.456529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6ef38-6d21-448f-8c64-69f38d5bf53a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:51.495779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:51.496393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:51.496670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:51.496941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:51.497570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] RBAC: Adding query filter params (type=image) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:51.497870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:51.499266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:51.509406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 521/1025] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:51.510477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:51.623829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89e4a6-4e6d-4aea-abf2-2076cc6e7076 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:51.659082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04122c-03a0-48b6-898a-b2d093b8c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:51.659708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04122c-03a0-48b6-898a-b2d093b8c8eb None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:51.659982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04122c-03a0-48b6-898a-b2d093b8c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:51.660247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04122c-03a0-48b6-898a-b2d093b8c8eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:51.660953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04122c-03a0-48b6-898a-b2d093b8c8eb None admin] RBAC: Adding query filter params (interface=public, service_id=65c5fcb1fb6e4dacb40826270640d6a4, region_id=RegionOne) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:51.661255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04122c-03a0-48b6-898a-b2d093b8c8eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:51.662659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04122c-03a0-48b6-898a-b2d093b8c8eb None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:51.669202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 505/1026] 10.222.0.94 () {56 vars in 1251 bytes} [Fri Aug 26 15:36:51 2022] GET /identity/v3/endpoints?service_id=65c5fcb1fb6e4dacb40826270640d6a4&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:51.674894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:51.782653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7423c4-baa0-404a-a4d1-9a937cdf1b4d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:51.817762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76d5bc-ca46-4f0e-95a5-7809d33972b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:51.818412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76d5bc-ca46-4f0e-95a5-7809d33972b0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:51.818700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76d5bc-ca46-4f0e-95a5-7809d33972b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:51.818980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76d5bc-ca46-4f0e-95a5-7809d33972b0 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:51.820136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76d5bc-ca46-4f0e-95a5-7809d33972b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:51.821530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76d5bc-ca46-4f0e-95a5-7809d33972b0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:51.834946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 522/1027] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:36:51 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:36:52.536564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e086f8-1cd2-4bf8-a3d0-0b015b593d2f None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:52.537110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e086f8-1cd2-4bf8-a3d0-0b015b593d2f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:52.537110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e086f8-1cd2-4bf8-a3d0-0b015b593d2f None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:52.539942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 506/1028] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:52.546023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432f8b9-3903-45f8-b5ea-fd5719464e8c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:52.546456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432f8b9-3903-45f8-b5ea-fd5719464e8c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:52.546456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432f8b9-3903-45f8-b5ea-fd5719464e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:52.570545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7432f8b9-3903-45f8-b5ea-fd5719464e8c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:52.572557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7432f8b9-3903-45f8-b5ea-fd5719464e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:52.587779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:52.651355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 523/1029] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:52.932389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e086f8-1cd2-4bf8-a3d0-0b015b593d2f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:52.996272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a2976-f251-4236-a222-92ef57b979af None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:52.996745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a2976-f251-4236-a222-92ef57b979af None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:52.997115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a2976-f251-4236-a222-92ef57b979af None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:52.998646 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 524/1030] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:53.001201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:53.004068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8827331-66bb-4aca-8ea4-f19df2dd46e2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.004344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8827331-66bb-4aca-8ea4-f19df2dd46e2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.004614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8827331-66bb-4aca-8ea4-f19df2dd46e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.023441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a8827331-66bb-4aca-8ea4-f19df2dd46e2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:53.024692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a8827331-66bb-4aca-8ea4-f19df2dd46e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:53.055959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e086f8-1cd2-4bf8-a3d0-0b015b593d2f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:53.099130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 525/1031] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:53.102332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173f9cf-dd43-4812-8521-1c1ec533e636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:53.103229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173f9cf-dd43-4812-8521-1c1ec533e636 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.103682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173f9cf-dd43-4812-8521-1c1ec533e636 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.104126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173f9cf-dd43-4812-8521-1c1ec533e636 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.115957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173f9cf-dd43-4812-8521-1c1ec533e636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-2173f9cf-dd43-4812-8521-1c1ec533e636 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:53.121769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:53.122669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:53.123541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:53.124166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 507/1032] 10.222.0.94 () {56 vars in 1120 bytes} [Fri Aug 26 15:36:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:53.130128 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8827331-66bb-4aca-8ea4-f19df2dd46e2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:53.213436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:53.248062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8827331-66bb-4aca-8ea4-f19df2dd46e2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:53.292497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:53.293457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.293928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.294413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.295303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:53.295817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:53.297384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:53.308110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 526/1033] 10.222.0.94 () {56 vars in 1115 bytes} [Fri Aug 26 15:36:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:53.313386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173f9cf-dd43-4812-8521-1c1ec533e636 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:53.386279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:53.423010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173f9cf-dd43-4812-8521-1c1ec533e636 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:53.439646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:53.466278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:53.466837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.467158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.467418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.478601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:53.480133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:53.497446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:53.498648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:53.499760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:53.499760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:53.499760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:53.499760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 508/1034] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:53.504296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:53.505942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc620362-9ca7-4b55-86ed-aab766338b6a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:53.548485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a166d-0191-430e-8ac5-3fe4a842d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:53.549162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a166d-0191-430e-8ac5-3fe4a842d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.549466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a166d-0191-430e-8ac5-3fe4a842d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.549810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a166d-0191-430e-8ac5-3fe4a842d5d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.550955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a166d-0191-430e-8ac5-3fe4a842d5d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:53.552396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a166d-0191-430e-8ac5-3fe4a842d5d1 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:53.559085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 527/1035] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:53.563645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a166d-0191-430e-8ac5-3fe4a842d5d1 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:53.564766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:53.628525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914766f3-c80f-4fe2-8489-0584d03af31c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:53.668716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:53.669329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.669607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.669876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.670520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:53.670817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:53.671994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a166d-0191-430e-8ac5-3fe4a842d5d1 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:53.672213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:53.686303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 509/1036] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:53.691809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:53.713427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec288270-7b24-4f8a-a585-76869efecb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:53.714053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec288270-7b24-4f8a-a585-76869efecb7b None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.714320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec288270-7b24-4f8a-a585-76869efecb7b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.714587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec288270-7b24-4f8a-a585-76869efecb7b None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.715549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec288270-7b24-4f8a-a585-76869efecb7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:53.717039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec288270-7b24-4f8a-a585-76869efecb7b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:53.722383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:53.803066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5e39e-3a27-46fd-87a3-bef8fed97dc9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:53.812848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 528/1037] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:36:53 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:36:53.813950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72245b5-3156-4920-b503-8b7d7890b174 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72245b5-3156-4920-b503-8b7d7890b174 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.814497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72245b5-3156-4920-b503-8b7d7890b174 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.815753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 529/1038] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:53.822105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2e06f-fb9e-4148-9e18-8973ff23d7e7 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.822394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2e06f-fb9e-4148-9e18-8973ff23d7e7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.822670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2e06f-fb9e-4148-9e18-8973ff23d7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.849034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2ab2e06f-fb9e-4148-9e18-8973ff23d7e7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:53.851642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2ab2e06f-fb9e-4148-9e18-8973ff23d7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:53.855137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:53.856394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:53.856890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:53.857252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:53.871328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:53.873722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:53.894216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:53.895517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:53.896473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 510/1039] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:36:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:53.902659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:53.938362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:53.951042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 530/1040] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:36:54.005659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:54.021887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2017c6e-89b7-42ce-a370-c2a69301053a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.058361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.059163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.059875 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.060472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.061776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:54.062114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.063933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.084492 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 511/1041] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:36:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:54.091310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2e06f-fb9e-4148-9e18-8973ff23d7e7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:54.198424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2e06f-fb9e-4148-9e18-8973ff23d7e7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.229397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:54.234503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d298e31-1a60-4917-9520-626a873b787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.235153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d298e31-1a60-4917-9520-626a873b787f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.235436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d298e31-1a60-4917-9520-626a873b787f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.235709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d298e31-1a60-4917-9520-626a873b787f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.305706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:54.316575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d298e31-1a60-4917-9520-626a873b787f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.318201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d298e31-1a60-4917-9520-626a873b787f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.342557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb689dc-1861-45eb-9138-372192c5c134 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.382836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.383515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.383794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.384057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] PATH_INFO: `/v3/services/image` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.384643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.386004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:54.394858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:54.395854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:54.396684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 512/1042] 10.222.0.94 () {56 vars in 1105 bytes} [Fri Aug 26 15:36:54 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:54.401980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:54.413668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:54.434361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 531/1043] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:36:54.440913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d298e31-1a60-4917-9520-626a873b787f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:54.518169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda93bb7-010f-4b14-abfe-a22f1739e48b None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.545084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:54.562087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d298e31-1a60-4917-9520-626a873b787f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.562692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.562692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.562692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.562692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.563058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:54.563058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.564877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.575555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 513/1044] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:54.580447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:54.603602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.604244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.604518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.604786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.617790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.619277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:54.634853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:54.635893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:54.636829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:54.636829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:54.636829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:54.636829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 532/1045] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:54.641653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:54.659304 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:54.690906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa94ab9-7d9d-47c1-a252-c616c8b001f2 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.729406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.730326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.730604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.730877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.731543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] RBAC: Adding query filter params (type=image) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:54.731843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.733206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.738475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 514/1046] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:54.744185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:54.756190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d0b40-7e3e-4869-8879-bcf9c712efc4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.796830 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.797497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.797780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.798045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.798687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:54.799047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.800465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.818419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 533/1047] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:54.820419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:54.823527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:54.852749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df23bd-4f3e-4c0c-a811-d730ebe77b65 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.892174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02378f-7950-4b4d-a365-750d575de12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.892782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02378f-7950-4b4d-a365-750d575de12f None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.893056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02378f-7950-4b4d-a365-750d575de12f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.893325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02378f-7950-4b4d-a365-750d575de12f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.893997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02378f-7950-4b4d-a365-750d575de12f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.895400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02378f-7950-4b4d-a365-750d575de12f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.902327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:54.938761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db58f8-1efa-4c4c-b59e-8d55d6e26d5a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:54.973994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:54.974621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:54.974894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:54.975191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:54.982495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:54.983994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:54.993392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 515/1048] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:36:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:55.003789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:55.004798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:55.005711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:55.005711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:55.005711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:55.005711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:55.005711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:55.005711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:55.005711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:55.005711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:55.011091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 534/1049] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:55.012157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02378f-7950-4b4d-a365-750d575de12f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:55.141589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02378f-7950-4b4d-a365-750d575de12f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:55.177987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395b99e-95cc-4106-9e0c-66d9829f83e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:55.178748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395b99e-95cc-4106-9e0c-66d9829f83e1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:55.179737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395b99e-95cc-4106-9e0c-66d9829f83e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:55.180152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395b99e-95cc-4106-9e0c-66d9829f83e1 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:55.180849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395b99e-95cc-4106-9e0c-66d9829f83e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:55.181150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395b99e-95cc-4106-9e0c-66d9829f83e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:55.182580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395b99e-95cc-4106-9e0c-66d9829f83e1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:55.195694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 516/1050] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:55.201702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:55.266693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:55.310716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82e74b-0d1a-4e5d-9334-0b924c5bfbc4 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:55.347507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752161a-d37a-4a03-8956-a12d6c6c0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:55.348115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752161a-d37a-4a03-8956-a12d6c6c0a1d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:55.348394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752161a-d37a-4a03-8956-a12d6c6c0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:55.348663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752161a-d37a-4a03-8956-a12d6c6c0a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:55.349392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752161a-d37a-4a03-8956-a12d6c6c0a1d None admin] RBAC: Adding query filter params (role.id=d6c6c174328447a29ce8d67c9cef9397, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=500d2f2bd83d499aa2814c6d7cc71795) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:55.349696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752161a-d37a-4a03-8956-a12d6c6c0a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:55.351768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752161a-d37a-4a03-8956-a12d6c6c0a1d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:55.358093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 535/1051] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:55 2022] GET /identity/v3/role_assignments?user.id=500d2f2bd83d499aa2814c6d7cc71795&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=d6c6c174328447a29ce8d67c9cef9397 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:55.888132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a988b7-e06d-4dd5-8c02-1eccc904de52 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:55.889170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a988b7-e06d-4dd5-8c02-1eccc904de52 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:55.889447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a988b7-e06d-4dd5-8c02-1eccc904de52 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:55.890820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 517/1052] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:55.896411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8e88-a537-4cc2-a33f-862bf91e3006 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:55.896865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8e88-a537-4cc2-a33f-862bf91e3006 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:55.896945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd8e88-a537-4cc2-a33f-862bf91e3006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:55.917052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7ffd8e88-a537-4cc2-a33f-862bf91e3006 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:55.918248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7ffd8e88-a537-4cc2-a33f-862bf91e3006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:55.995214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 536/1053] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:56.270185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a988b7-e06d-4dd5-8c02-1eccc904de52 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:56.325223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:56.380094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a988b7-e06d-4dd5-8c02-1eccc904de52 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:56.418112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552475e-28d7-4a62-8ee4-0e33e2aa1b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:56.418761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552475e-28d7-4a62-8ee4-0e33e2aa1b43 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:56.419076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552475e-28d7-4a62-8ee4-0e33e2aa1b43 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:56.419341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552475e-28d7-4a62-8ee4-0e33e2aa1b43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:56.430973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552475e-28d7-4a62-8ee4-0e33e2aa1b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-9552475e-28d7-4a62-8ee4-0e33e2aa1b43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:56.435814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:56.436744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:56.437571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 518/1054] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:56.446954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd8e88-a537-4cc2-a33f-862bf91e3006 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:56.515706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:56.571893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd8e88-a537-4cc2-a33f-862bf91e3006 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:56.608898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:56.609507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:56.609784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:56.610051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:56.610707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:56.611032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:56.612389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:56.623112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 537/1055] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:56.628718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552475e-28d7-4a62-8ee4-0e33e2aa1b43 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:56.732957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552475e-28d7-4a62-8ee4-0e33e2aa1b43 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:56.743150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:56.768423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a69aab-65fb-4756-8cdb-3f5a261fde8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:56.769152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a69aab-65fb-4756-8cdb-3f5a261fde8f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:56.769501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a69aab-65fb-4756-8cdb-3f5a261fde8f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:56.769697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a69aab-65fb-4756-8cdb-3f5a261fde8f None admin] PATH_INFO: `/v3/users/2fda18b39e5c4fbea18e29f9a02a4879` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:56.784817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a69aab-65fb-4756-8cdb-3f5a261fde8f None admin] RBAC: Authorizing `identity:get_user(user_id=2fda18b39e5c4fbea18e29f9a02a4879)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:56.786279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a69aab-65fb-4756-8cdb-3f5a261fde8f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:56.793355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 519/1056] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:36:56 2022] GET /identity/v3/users/2fda18b39e5c4fbea18e29f9a02a4879 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:56.794766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:56.899972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dba09-750e-4526-9f64-fe698db68e07 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:56.921009 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:56.938739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dde5b-f1ec-4d2a-9c96-d0b11c07e028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:56.939396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dde5b-f1ec-4d2a-9c96-d0b11c07e028 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:56.939666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dde5b-f1ec-4d2a-9c96-d0b11c07e028 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:56.939929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dde5b-f1ec-4d2a-9c96-d0b11c07e028 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:56.951195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2130e-6b1b-49dc-a59a-d7990a0da481 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:56.951714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2130e-6b1b-49dc-a59a-d7990a0da481 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:56.951714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2130e-6b1b-49dc-a59a-d7990a0da481 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:56.953489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 520/1057] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:56.958381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b433a3b-43d9-4ac6-8c17-4b48cf040bcf None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:56.958662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b433a3b-43d9-4ac6-8c17-4b48cf040bcf None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:56.958941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b433a3b-43d9-4ac6-8c17-4b48cf040bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:56.977825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-2b433a3b-43d9-4ac6-8c17-4b48cf040bcf None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:56.979067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2b433a3b-43d9-4ac6-8c17-4b48cf040bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:57.018156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dde5b-f1ec-4d2a-9c96-d0b11c07e028 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:57.019726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dde5b-f1ec-4d2a-9c96-d0b11c07e028 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:57.063288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 521/1058] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:57.133254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 538/1059] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:36:57.139184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b433a3b-43d9-4ac6-8c17-4b48cf040bcf None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:57.238344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:57.245134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b433a3b-43d9-4ac6-8c17-4b48cf040bcf None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:57.283453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60442d40-aa8a-4ec9-a384-95ece209aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:57.284088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442d40-aa8a-4ec9-a384-95ece209aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:57.284365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442d40-aa8a-4ec9-a384-95ece209aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:57.284637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442d40-aa8a-4ec9-a384-95ece209aaf1 None admin] PATH_INFO: `/v3/projects/01d68589735f44108dbf04bc2c9b762b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:57.285768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60442d40-aa8a-4ec9-a384-95ece209aaf1 None admin] RBAC: Authorizing `identity:get_project(project_id=01d68589735f44108dbf04bc2c9b762b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:57.287234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60442d40-aa8a-4ec9-a384-95ece209aaf1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:57.294367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 522/1060] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:36:57 2022] GET /identity/v3/projects/01d68589735f44108dbf04bc2c9b762b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:57.295549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dde5b-f1ec-4d2a-9c96-d0b11c07e028 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:57.315821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:57.316461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc86e11-5424-439e-b38b-0396f2be41e5 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:57.316742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc86e11-5424-439e-b38b-0396f2be41e5 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:57.317010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc86e11-5424-439e-b38b-0396f2be41e5 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:57.318474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 523/1061] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:36:57.324675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1522ac4-f53d-4596-abdc-bc9d61559340 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:57.324950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1522ac4-f53d-4596-abdc-bc9d61559340 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:57.325213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1522ac4-f53d-4596-abdc-bc9d61559340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:57.344124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c1522ac4-f53d-4596-abdc-bc9d61559340 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:57.345312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c1522ac4-f53d-4596-abdc-bc9d61559340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:57.404340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dde5b-f1ec-4d2a-9c96-d0b11c07e028 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:57.415529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 524/1062] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:57.416384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1522ac4-f53d-4596-abdc-bc9d61559340 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:57.442492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:57.443130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:57.443401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:57.443661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:57.444988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=01d68589735f44108dbf04bc2c9b762b, user.id=2fda18b39e5c4fbea18e29f9a02a4879) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:57.445290 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:57.446891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:57.453086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 539/1063] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:36:57 2022] GET /identity/v3/role_assignments?user.id=2fda18b39e5c4fbea18e29f9a02a4879&scope.project.id=01d68589735f44108dbf04bc2c9b762b&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:57.529468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:57.537312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1522ac4-f53d-4596-abdc-bc9d61559340 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:57.578957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce3c24-a6a8-4e52-9c7b-62e5cc67f960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:57.579594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce3c24-a6a8-4e52-9c7b-62e5cc67f960 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:57.579869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce3c24-a6a8-4e52-9c7b-62e5cc67f960 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:57.580139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce3c24-a6a8-4e52-9c7b-62e5cc67f960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:57.654840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce3c24-a6a8-4e52-9c7b-62e5cc67f960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:57.656319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce3c24-a6a8-4e52-9c7b-62e5cc67f960 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:57.691582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:57.761157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:57.771078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 525/1064] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:36:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:36:57.776839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce3c24-a6a8-4e52-9c7b-62e5cc67f960 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:57.795866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:57.818421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372a71b-fbdf-40dd-be85-a369b1323354 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:57.859419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadebda1-0d3a-4ce2-be61-4e78220da0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:57.860053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadebda1-0d3a-4ce2-be61-4e78220da0a3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:57.860389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadebda1-0d3a-4ce2-be61-4e78220da0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:57.860590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadebda1-0d3a-4ce2-be61-4e78220da0a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:57.873159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadebda1-0d3a-4ce2-be61-4e78220da0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-dadebda1-0d3a-4ce2-be61-4e78220da0a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:57.877866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:57.878807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:57.879723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 540/1065] 10.222.0.94 () {56 vars in 1120 bytes} [Fri Aug 26 15:36:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:57.885346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadebda1-0d3a-4ce2-be61-4e78220da0a3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:57.893456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce3c24-a6a8-4e52-9c7b-62e5cc67f960 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:57.934464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:57.935133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:57.935422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:57.935691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:57.948759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:57.950557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:57.968577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:57.970029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:57.970973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:57.970973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:57.970973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:36:57.970973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 526/1066] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:57.975746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:57.995352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadebda1-0d3a-4ce2-be61-4e78220da0a3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:58.029328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:58.031627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:58.032904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:58.032904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:58.032904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:58.033402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:58.033699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:58.035082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:58.049418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 541/1067] 10.222.0.94 () {56 vars in 1115 bytes} [Fri Aug 26 15:36:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:36:58.050225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:58.055472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:58.097026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347399a-ebba-4fe2-9f94-49ca281c4ced None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:58.134331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:58.134989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:58.135250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:58.135523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:58.136167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:58.136467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:58.137831 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:58.150292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 527/1068] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:58.158758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7377f-eb91-4b80-ad60-b2bacf73e046 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:58.198266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:58.198954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:58.199219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:58.199481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:58.212073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:58.213565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:58.229692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:58.230730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:58.236070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:58.236070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:58.236070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:58.236070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 542/1069] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:36:58.238541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:58.348392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fd62d-9fc9-4b4d-bc1e-368e74b7a110 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:58.357222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:58.388197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:58.388807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:58.389085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:58.389355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:58.390067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:58.390784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:58.392479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:58.406373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 528/1070] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:58.412145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:58.478259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:58.525833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273f9d-a66a-4ffe-b494-0ad7511e0ede None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:58.564053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:58.564959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:58.565386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:58.565811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:58.574649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:58.576130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:36:58.588552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:58.589584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:58.590574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 543/1071] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:36:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:58.595493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:58.701919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c621ea-cdde-4341-933c-080757a7bc12 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:58.736804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:58.737403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:58.737678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:58.737946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:58.738668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:58.738994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:58.740679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:58.758816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 529/1072] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:36:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:58.764399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:58.865370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31c8ef-1bef-4718-b1cf-f58b565c7652 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:58.883466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:58.907972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:58.908609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:58.908882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:58.909152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:58.920369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:58.921813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:36:58.937438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:58.938612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:36:58.939558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:36:58.939558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:36:58.939558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:58.939558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 544/1073] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:58.944736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:58.970668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:59.048511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e257d-c4b4-4c23-b092-f8e876c9c34d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:59.082414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:59.083047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:59.083329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:59.083597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:59.084232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:59.084528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:59.086397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:59.097555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 530/1074] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:59.104166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:59.211913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6e0da-3b9d-47ac-b32c-fd727f177d3d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:59.245967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79f02c-215d-4d17-b463-75efcb25fc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:59.246512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79f02c-215d-4d17-b463-75efcb25fc28 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:59.246784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79f02c-215d-4d17-b463-75efcb25fc28 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:59.247083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79f02c-215d-4d17-b463-75efcb25fc28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:59.253643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79f02c-215d-4d17-b463-75efcb25fc28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:59.255087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79f02c-215d-4d17-b463-75efcb25fc28 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-0f79f02c-215d-4d17-b463-75efcb25fc28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:36:59.266141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:59.267075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:59.267954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 545/1075] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:36:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:59.273718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:59.360267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:59.371495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b347d9-7c52-43e6-a5c0-872fa0836f5c None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:59.376426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b347d9-7c52-43e6-a5c0-872fa0836f5c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:59.376426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b347d9-7c52-43e6-a5c0-872fa0836f5c None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:59.376426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 546/1076] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:36:59.376691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890ca2-981e-4510-a9b1-55f3f0c31f11 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:59.378135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34875c-77c3-4162-84ad-a13d8aaa1ff9 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:59.378414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34875c-77c3-4162-84ad-a13d8aaa1ff9 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:59.378686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34875c-77c3-4162-84ad-a13d8aaa1ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:59.397535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ed34875c-77c3-4162-84ad-a13d8aaa1ff9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:36:59.398724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ed34875c-77c3-4162-84ad-a13d8aaa1ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:36:59.411498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:59.412103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:59.412376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:59.412643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:59.414286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:36:59.414862 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:59.416655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:59.422327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:36:59.429913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 531/1077] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:36:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:36:59.435968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:59.476107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 547/1078] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:36:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:36:59.539644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e2194-ef13-4dd1-ae08-85d0b8c25d4e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:59.577531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f2c28-2a4a-44e0-bc67-2108ffd068f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:59.578199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2c28-2a4a-44e0-bc67-2108ffd068f9 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:59.578472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2c28-2a4a-44e0-bc67-2108ffd068f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:59.578744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f2c28-2a4a-44e0-bc67-2108ffd068f9 None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/500d2f2bd83d499aa2814c6d7cc71795/roles/d6c6c174328447a29ce8d67c9cef9397` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:59.581225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f2c28-2a4a-44e0-bc67-2108ffd068f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=500d2f2bd83d499aa2814c6d7cc71795, role_id=d6c6c174328447a29ce8d67c9cef9397)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:36:59.582667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f2c28-2a4a-44e0-bc67-2108ffd068f9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:36:59.669360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 532/1079] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:36:59 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/500d2f2bd83d499aa2814c6d7cc71795/roles/d6c6c174328447a29ce8d67c9cef9397 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:36:59.748012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34875c-77c3-4162-84ad-a13d8aaa1ff9 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:59.851906 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34875c-77c3-4162-84ad-a13d8aaa1ff9 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:36:59.885073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c9cb3-7a9a-4eb3-ad23-b5bf19915f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:36:59.885704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9cb3-7a9a-4eb3-ad23-b5bf19915f91 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:36:59.885976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9cb3-7a9a-4eb3-ad23-b5bf19915f91 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:36:59.886240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9cb3-7a9a-4eb3-ad23-b5bf19915f91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:36:59.896191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c9cb3-7a9a-4eb3-ad23-b5bf19915f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-275c9cb3-7a9a-4eb3-ad23-b5bf19915f91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:36:59.901456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:36:59.902391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:36:59.903279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:36:59.907079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 548/1080] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:36:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:36:59.908494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f2c28-2a4a-44e0-bc67-2108ffd068f9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:36:59.912393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:00.016022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f2c28-2a4a-44e0-bc67-2108ffd068f9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:00.054363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:00.055003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:00.055286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:00.055560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:00.056224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:00.056523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:00.057888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:00.066897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 533/1081] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:36:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:00.073394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c9cb3-7a9a-4eb3-ad23-b5bf19915f91 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:00.095489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:00.180427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c9cb3-7a9a-4eb3-ad23-b5bf19915f91 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:00.216695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063286b-8489-4430-bbb5-281eff67b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:00.217331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063286b-8489-4430-bbb5-281eff67b565 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:00.217607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063286b-8489-4430-bbb5-281eff67b565 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:00.217874 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063286b-8489-4430-bbb5-281eff67b565 None admin] PATH_INFO: `/v3/users/2fda18b39e5c4fbea18e29f9a02a4879` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:00.219016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063286b-8489-4430-bbb5-281eff67b565 None admin] RBAC: Authorizing `identity:get_user(user_id=2fda18b39e5c4fbea18e29f9a02a4879)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:00.220449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063286b-8489-4430-bbb5-281eff67b565 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:00.222022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 549/1082] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:00 2022] GET /identity/v3/users/2fda18b39e5c4fbea18e29f9a02a4879 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:00.228076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:00.295415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:00.330822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10751b7-35c3-4973-8eb0-851a80e28eb0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:00.364698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777b3f4-488d-47e1-9d2d-d1c03d511c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:00.365325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b3f4-488d-47e1-9d2d-d1c03d511c34 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:00.365594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b3f4-488d-47e1-9d2d-d1c03d511c34 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:00.365941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b3f4-488d-47e1-9d2d-d1c03d511c34 None admin] PATH_INFO: `/v3/projects/01d68589735f44108dbf04bc2c9b762b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:00.367184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777b3f4-488d-47e1-9d2d-d1c03d511c34 None admin] RBAC: Authorizing `identity:get_project(project_id=01d68589735f44108dbf04bc2c9b762b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:00.368595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777b3f4-488d-47e1-9d2d-d1c03d511c34 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:00.370218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 534/1083] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:00 2022] GET /identity/v3/projects/01d68589735f44108dbf04bc2c9b762b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:00.376092 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063286b-8489-4430-bbb5-281eff67b565 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:00.462183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:00.480802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063286b-8489-4430-bbb5-281eff67b565 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:00.517814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86edb2c-4629-40ae-aec2-1f7d18670722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:00.518476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86edb2c-4629-40ae-aec2-1f7d18670722 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:00.518750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86edb2c-4629-40ae-aec2-1f7d18670722 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:00.519046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86edb2c-4629-40ae-aec2-1f7d18670722 None admin] PATH_INFO: `/v3/projects/01d68589735f44108dbf04bc2c9b762b/users/2fda18b39e5c4fbea18e29f9a02a4879/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:00.520991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86edb2c-4629-40ae-aec2-1f7d18670722 None admin] RBAC: Authorizing `identity:create_grant(project_id=01d68589735f44108dbf04bc2c9b762b, user_id=2fda18b39e5c4fbea18e29f9a02a4879, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:00.522768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86edb2c-4629-40ae-aec2-1f7d18670722 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:00.597174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 550/1084] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:37:00 2022] PUT /identity/v3/projects/01d68589735f44108dbf04bc2c9b762b/users/2fda18b39e5c4fbea18e29f9a02a4879/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:37:01.613635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be2bc-c42b-467d-b50f-e69ca96b6325 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:01.614451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be2bc-c42b-467d-b50f-e69ca96b6325 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:01.614451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be2bc-c42b-467d-b50f-e69ca96b6325 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:01.615925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 535/1085] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:01.621448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb581a56-6957-4b78-b5df-85628bcf70be None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:01.621887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb581a56-6957-4b78-b5df-85628bcf70be None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:01.621982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb581a56-6957-4b78-b5df-85628bcf70be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:01.640614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-fb581a56-6957-4b78-b5df-85628bcf70be None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:01.641810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-fb581a56-6957-4b78-b5df-85628bcf70be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:01.711997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 551/1086] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:01.982815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5be2bc-c42b-467d-b50f-e69ca96b6325 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:02.085262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5be2bc-c42b-467d-b50f-e69ca96b6325 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:02.119921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff511ce3-fce2-4f90-a36c-28ca2bb96177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:02.120552 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff511ce3-fce2-4f90-a36c-28ca2bb96177 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:02.120831 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff511ce3-fce2-4f90-a36c-28ca2bb96177 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:02.121100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff511ce3-fce2-4f90-a36c-28ca2bb96177 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:02.123036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:02.132466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff511ce3-fce2-4f90-a36c-28ca2bb96177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ff511ce3-fce2-4f90-a36c-28ca2bb96177 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:37:02.138880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:02.140032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:37:02.140863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 536/1087] 10.222.0.94 () {56 vars in 1120 bytes} [Fri Aug 26 15:37:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:02.144401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb581a56-6957-4b78-b5df-85628bcf70be None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:02.223041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:02.247616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb581a56-6957-4b78-b5df-85628bcf70be None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:02.283810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:02.284418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:02.284813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:02.285085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:02.285753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:02.286052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:02.287501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:02.297004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 552/1088] 10.222.0.94 () {56 vars in 1115 bytes} [Fri Aug 26 15:37:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:02.302700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff511ce3-fce2-4f90-a36c-28ca2bb96177 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:02.403244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff511ce3-fce2-4f90-a36c-28ca2bb96177 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:02.438759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940347-31d8-4359-bfc3-0a6a12fe7411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:02.439422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940347-31d8-4359-bfc3-0a6a12fe7411 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:02.439699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940347-31d8-4359-bfc3-0a6a12fe7411 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:02.439969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940347-31d8-4359-bfc3-0a6a12fe7411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:02.450305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940347-31d8-4359-bfc3-0a6a12fe7411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:02.451781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940347-31d8-4359-bfc3-0a6a12fe7411 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-5c940347-31d8-4359-bfc3-0a6a12fe7411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:37:02.466305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:02.469970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:37:02.470883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:37:02.470883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:37:02.470883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:37:02.470883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 537/1089] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:02.473487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:02.549192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcca6f-fc34-4184-bef1-9901e5556dc3 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:02.549393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcca6f-fc34-4184-bef1-9901e5556dc3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:02.549684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcca6f-fc34-4184-bef1-9901e5556dc3 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:02.551099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 538/1090] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:37:02.557643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053a12f-baba-469f-bf65-85c62ff6dd6e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:02.557943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053a12f-baba-469f-bf65-85c62ff6dd6e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:02.558189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053a12f-baba-469f-bf65-85c62ff6dd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:02.577010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0053a12f-baba-469f-bf65-85c62ff6dd6e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:02.578876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0053a12f-baba-469f-bf65-85c62ff6dd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:02.585148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f15575-706a-4ee1-938f-b0b5dfdb306c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:02.621268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:02.621886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:02.622534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:02.622534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:02.623163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:02.623466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:02.624383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:02.625039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:02.632929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:02.638400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 553/1091] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:02.644112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:02.649514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 539/1092] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:37:02.743996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3fa82-0e89-4487-85ad-d0bd139e8c44 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:02.778362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:02.779087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:02.779393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:02.779687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:02.786750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:02.788903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:37:02.800663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:02.802055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:37:02.803135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 554/1093] 10.222.0.94 () {56 vars in 1093 bytes} [Fri Aug 26 15:37:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:37:02.807505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053a12f-baba-469f-bf65-85c62ff6dd6e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:02.905955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053a12f-baba-469f-bf65-85c62ff6dd6e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:02.919862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:02.942278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:02.942889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:02.943197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:02.943470 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:02.945055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:02.945463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:02.946896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:02.964423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 540/1094] 10.222.0.94 () {56 vars in 1133 bytes} [Fri Aug 26 15:37:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:02.970119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:02.983811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.008281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.027494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee59ba9-1605-484a-95c7-15c78213a9e7 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.064870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d266a-bc06-45e0-bd02-9879de8f7fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.065430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d266a-bc06-45e0-bd02-9879de8f7fba None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.065703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d266a-bc06-45e0-bd02-9879de8f7fba None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.065965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d266a-bc06-45e0-bd02-9879de8f7fba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.076908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d266a-bc06-45e0-bd02-9879de8f7fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:37:03.077074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64334c28-31d6-496a-926c-17bd66618e06 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-b21d266a-bc06-45e0-bd02-9879de8f7fba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:37:03.082116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:03.083088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:37:03.083924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 555/1095] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:37:03.089874 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d266a-bc06-45e0-bd02-9879de8f7fba None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.115115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc462e-d0f8-4d02-b3cc-2ab363852ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.115720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc462e-d0f8-4d02-b3cc-2ab363852ab2 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.116814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc462e-d0f8-4d02-b3cc-2ab363852ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.117309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc462e-d0f8-4d02-b3cc-2ab363852ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.189954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc462e-d0f8-4d02-b3cc-2ab363852ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.191439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc462e-d0f8-4d02-b3cc-2ab363852ab2 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.193169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d266a-bc06-45e0-bd02-9879de8f7fba None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.216379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.237973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.238658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.241664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.241664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.241664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:03.241664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.242460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.242889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.252682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 556/1096] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:03.258786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.314856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 541/1097] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:37:03.320963 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc462e-d0f8-4d02-b3cc-2ab363852ab2 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.362304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6770a3-fd64-45e7-9136-c6adfe5723d3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.399299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3c812-de96-4e43-8474-116a44870d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.399940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c812-de96-4e43-8474-116a44870d22 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.400211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c812-de96-4e43-8474-116a44870d22 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.400477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c812-de96-4e43-8474-116a44870d22 None admin] PATH_INFO: `/v3/users/2fda18b39e5c4fbea18e29f9a02a4879` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.401614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3c812-de96-4e43-8474-116a44870d22 None admin] RBAC: Authorizing `identity:get_user(user_id=2fda18b39e5c4fbea18e29f9a02a4879)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.403063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3c812-de96-4e43-8474-116a44870d22 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.404643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 557/1098] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/users/2fda18b39e5c4fbea18e29f9a02a4879 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:03.410386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3c812-de96-4e43-8474-116a44870d22 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.427291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc462e-d0f8-4d02-b3cc-2ab363852ab2 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.462030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.463984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.464642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.464914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.465180 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.467234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.480942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.482436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:37:03.497667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:03.498873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:37:03.499798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:37:03.499798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:37:03.499798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:37:03.499798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 542/1099] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:37:03.504718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.513829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3c812-de96-4e43-8474-116a44870d22 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.551751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60fb82-3faf-4cd6-88ec-b099fceac59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.552474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60fb82-3faf-4cd6-88ec-b099fceac59b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.552822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60fb82-3faf-4cd6-88ec-b099fceac59b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.553315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60fb82-3faf-4cd6-88ec-b099fceac59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.613124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135662cd-eebf-47b5-81ea-19b679470fa9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.632737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60fb82-3faf-4cd6-88ec-b099fceac59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.634406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60fb82-3faf-4cd6-88ec-b099fceac59b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.649896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.650501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.650772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.651074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.651711 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:03.652114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.653394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.666063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 543/1100] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:03.671673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.685264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.718988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.739289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 558/1101] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:37:03.745220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60fb82-3faf-4cd6-88ec-b099fceac59b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.774006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b4bcf-8f04-48da-ac18-aafb66ef2d86 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.807846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.808471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.808743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.809007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.815521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.816934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:37:03.828424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:03.829409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:37:03.830292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 544/1102] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:37:03.835807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.845654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60fb82-3faf-4cd6-88ec-b099fceac59b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.880077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1a96-2e06-4c80-bf2c-fe8044952f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.880708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1a96-2e06-4c80-bf2c-fe8044952f0d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.880985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1a96-2e06-4c80-bf2c-fe8044952f0d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.881251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1a96-2e06-4c80-bf2c-fe8044952f0d None admin] PATH_INFO: `/v3/projects/01d68589735f44108dbf04bc2c9b762b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.883123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f1a96-2e06-4c80-bf2c-fe8044952f0d None admin] RBAC: Authorizing `identity:get_project(project_id=01d68589735f44108dbf04bc2c9b762b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.884555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f1a96-2e06-4c80-bf2c-fe8044952f0d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.891183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 559/1103] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/projects/01d68589735f44108dbf04bc2c9b762b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:03.892140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1a96-2e06-4c80-bf2c-fe8044952f0d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.910839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.936011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdaf25e-f35a-4201-9085-948d7665dd7e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:03.943040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:03.970818 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:03.971457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:03.971732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:03.972000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:03.972744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:03.973306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:03.975494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:03.988832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 545/1104] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:03.995517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:03.999741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1a96-2e06-4c80-bf2c-fe8044952f0d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:04.034895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8a804-4411-4bc1-a557-d21f317691c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:04.035574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8a804-4411-4bc1-a557-d21f317691c6 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:04.035852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8a804-4411-4bc1-a557-d21f317691c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:04.036117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8a804-4411-4bc1-a557-d21f317691c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:04.036831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8a804-4411-4bc1-a557-d21f317691c6 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=01d68589735f44108dbf04bc2c9b762b, user.id=2fda18b39e5c4fbea18e29f9a02a4879) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:04.037130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8a804-4411-4bc1-a557-d21f317691c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:04.039165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8a804-4411-4bc1-a557-d21f317691c6 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:04.045239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 560/1105] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/role_assignments?user.id=2fda18b39e5c4fbea18e29f9a02a4879&scope.project.id=01d68589735f44108dbf04bc2c9b762b&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:04.104291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1069b-8996-4ec2-9474-81076628ac60 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:04.142362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed187d2-5352-4fac-b452-2681d8c65212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:04.143026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed187d2-5352-4fac-b452-2681d8c65212 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:04.143296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed187d2-5352-4fac-b452-2681d8c65212 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:04.143568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed187d2-5352-4fac-b452-2681d8c65212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:04.144308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed187d2-5352-4fac-b452-2681d8c65212 None admin] RBAC: Adding query filter params (role.id=d6c6c174328447a29ce8d67c9cef9397, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=500d2f2bd83d499aa2814c6d7cc71795) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:04.144612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed187d2-5352-4fac-b452-2681d8c65212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:04.146068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed187d2-5352-4fac-b452-2681d8c65212 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:04.153080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 546/1106] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:37:03 2022] GET /identity/v3/role_assignments?user.id=500d2f2bd83d499aa2814c6d7cc71795&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=d6c6c174328447a29ce8d67c9cef9397 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:06.048667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92f77-d8ac-43c7-a2f9-0dd8ead37ffb None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:06.049259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92f77-d8ac-43c7-a2f9-0dd8ead37ffb None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:06.049259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92f77-d8ac-43c7-a2f9-0dd8ead37ffb None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:06.050568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 561/1107] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:06.056822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf2913-dfeb-49a5-aa93-f0fa527c70dc None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:06.057304 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf2913-dfeb-49a5-aa93-f0fa527c70dc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:06.057409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf2913-dfeb-49a5-aa93-f0fa527c70dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:06.075288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-70bf2913-dfeb-49a5-aa93-f0fa527c70dc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:06.076486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-70bf2913-dfeb-49a5-aa93-f0fa527c70dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:06.100529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:06.145036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 547/1108] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:06.417385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa92f77-d8ac-43c7-a2f9-0dd8ead37ffb None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:06.501899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:06.521889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa92f77-d8ac-43c7-a2f9-0dd8ead37ffb None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:06.556425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8fba0-336c-4952-abea-252131becdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:06.557046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8fba0-336c-4952-abea-252131becdb1 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:06.557315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8fba0-336c-4952-abea-252131becdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:06.557576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8fba0-336c-4952-abea-252131becdb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:06.558668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8fba0-336c-4952-abea-252131becdb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:06.560111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8fba0-336c-4952-abea-252131becdb1 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:06.561712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 562/1109] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:37:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:06.567602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf2913-dfeb-49a5-aa93-f0fa527c70dc None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:06.666824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf2913-dfeb-49a5-aa93-f0fa527c70dc None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:06.701030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5fcee-adc5-4437-92b2-20c679c1c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:06.701557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5fcee-adc5-4437-92b2-20c679c1c7c6 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:06.701829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5fcee-adc5-4437-92b2-20c679c1c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:06.702120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5fcee-adc5-4437-92b2-20c679c1c7c6 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:06.702752 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5fcee-adc5-4437-92b2-20c679c1c7c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:06.704199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5fcee-adc5-4437-92b2-20c679c1c7c6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:06.795420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 548/1110] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:37:06 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:37:08.729416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258588dd-790d-49ed-8835-d9a3bcaa23d5 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:08.729975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258588dd-790d-49ed-8835-d9a3bcaa23d5 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:08.729975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258588dd-790d-49ed-8835-d9a3bcaa23d5 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:08.731329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 563/1111] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:08.736415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88bb56-9067-4d9f-908c-30d6b98d46d9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:08.736821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88bb56-9067-4d9f-908c-30d6b98d46d9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:08.736929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88bb56-9067-4d9f-908c-30d6b98d46d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:08.754935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7d88bb56-9067-4d9f-908c-30d6b98d46d9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:08.756158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7d88bb56-9067-4d9f-908c-30d6b98d46d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:08.767026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:08.822090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 549/1112] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:09.094233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258588dd-790d-49ed-8835-d9a3bcaa23d5 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:09.197003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258588dd-790d-49ed-8835-d9a3bcaa23d5 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:09.230377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40074cd5-2446-4129-82e5-4aa8a15f200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:09.231036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40074cd5-2446-4129-82e5-4aa8a15f200b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:09.231315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40074cd5-2446-4129-82e5-4aa8a15f200b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:09.231583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40074cd5-2446-4129-82e5-4aa8a15f200b None admin] PATH_INFO: `/v3/roles/98aaf5af7bb24b1596a9b01fd76eaf2c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:09.232966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40074cd5-2446-4129-82e5-4aa8a15f200b None admin] RBAC: Authorizing `identity:get_role(role_id=98aaf5af7bb24b1596a9b01fd76eaf2c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:09.234321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40074cd5-2446-4129-82e5-4aa8a15f200b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:09.235659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 564/1113] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:09 2022] GET /identity/v3/roles/98aaf5af7bb24b1596a9b01fd76eaf2c => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:09.241852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88bb56-9067-4d9f-908c-30d6b98d46d9 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:09.337705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88bb56-9067-4d9f-908c-30d6b98d46d9 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:09.370606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de504d-96d2-43a9-b1a4-93531adaa135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:09.371255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de504d-96d2-43a9-b1a4-93531adaa135 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:09.371526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de504d-96d2-43a9-b1a4-93531adaa135 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:09.371792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de504d-96d2-43a9-b1a4-93531adaa135 None admin] PATH_INFO: `/v3/users/0aa73242cbdb4e01b165cb05a4510c5b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:09.385269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de504d-96d2-43a9-b1a4-93531adaa135 None admin] RBAC: Authorizing `identity:get_user(user_id=0aa73242cbdb4e01b165cb05a4510c5b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:09.386702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de504d-96d2-43a9-b1a4-93531adaa135 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:09.388322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 550/1114] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:09 2022] GET /identity/v3/users/0aa73242cbdb4e01b165cb05a4510c5b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:09.394412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40074cd5-2446-4129-82e5-4aa8a15f200b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:09.414286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:09.502472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40074cd5-2446-4129-82e5-4aa8a15f200b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:09.536738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7600ab5-28ae-465b-9ead-368644d93248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:09.540275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7600ab5-28ae-465b-9ead-368644d93248 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:09.540275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7600ab5-28ae-465b-9ead-368644d93248 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:09.540275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7600ab5-28ae-465b-9ead-368644d93248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:09.606159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7600ab5-28ae-465b-9ead-368644d93248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:09.607851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7600ab5-28ae-465b-9ead-368644d93248 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:09.625418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:09.708127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 565/1115] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:37:09.714326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de504d-96d2-43a9-b1a4-93531adaa135 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:09.718162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:09.815949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de504d-96d2-43a9-b1a4-93531adaa135 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:09.850296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b4dc-9c48-4cf3-beb9-df66faa12532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:09.850952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4dc-9c48-4cf3-beb9-df66faa12532 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:09.851237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4dc-9c48-4cf3-beb9-df66faa12532 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:09.851504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4dc-9c48-4cf3-beb9-df66faa12532 None admin] PATH_INFO: `/v3/projects/01d68589735f44108dbf04bc2c9b762b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:09.852640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b4dc-9c48-4cf3-beb9-df66faa12532 None admin] RBAC: Authorizing `identity:get_project(project_id=01d68589735f44108dbf04bc2c9b762b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:09.854079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b4dc-9c48-4cf3-beb9-df66faa12532 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:09.855765 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 551/1116] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:09 2022] GET /identity/v3/projects/01d68589735f44108dbf04bc2c9b762b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:09.862665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7600ab5-28ae-465b-9ead-368644d93248 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:09.970123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7600ab5-28ae-465b-9ead-368644d93248 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:09.987496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:10.007152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550298c9-0b77-412f-b10b-69598b6b37bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:10.007907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550298c9-0b77-412f-b10b-69598b6b37bd None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:10.008202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550298c9-0b77-412f-b10b-69598b6b37bd None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:10.008511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550298c9-0b77-412f-b10b-69598b6b37bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:10.009248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550298c9-0b77-412f-b10b-69598b6b37bd None admin] RBAC: Adding query filter params (role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=01d68589735f44108dbf04bc2c9b762b, user.id=0aa73242cbdb4e01b165cb05a4510c5b) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:10.009570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550298c9-0b77-412f-b10b-69598b6b37bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:10.011166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550298c9-0b77-412f-b10b-69598b6b37bd None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:10.016739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 566/1117] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:37:09 2022] GET /identity/v3/role_assignments?user.id=0aa73242cbdb4e01b165cb05a4510c5b&scope.project.id=01d68589735f44108dbf04bc2c9b762b&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:11.887865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5a59d-0932-47b0-a4f3-24a31078208f None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:11.888405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5a59d-0932-47b0-a4f3-24a31078208f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:11.888405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5a59d-0932-47b0-a4f3-24a31078208f None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:11.889720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 552/1118] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:11.894807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c33d4-0e7b-4fff-abee-727faefeb3e1 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:11.895266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c33d4-0e7b-4fff-abee-727faefeb3e1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:11.895372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c33d4-0e7b-4fff-abee-727faefeb3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:11.913771 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1d3c33d4-0e7b-4fff-abee-727faefeb3e1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:11.915003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1d3c33d4-0e7b-4fff-abee-727faefeb3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:11.984571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 567/1119] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:12.267973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5a59d-0932-47b0-a4f3-24a31078208f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:12.340902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:12.371986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5a59d-0932-47b0-a4f3-24a31078208f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:12.406562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bd623-c05e-4ad6-ad56-9051ceccfc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:12.407209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bd623-c05e-4ad6-ad56-9051ceccfc14 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:12.407496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bd623-c05e-4ad6-ad56-9051ceccfc14 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:12.407761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bd623-c05e-4ad6-ad56-9051ceccfc14 None admin] PATH_INFO: `/v3/roles/98aaf5af7bb24b1596a9b01fd76eaf2c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:12.409155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bd623-c05e-4ad6-ad56-9051ceccfc14 None admin] RBAC: Authorizing `identity:get_role(role_id=98aaf5af7bb24b1596a9b01fd76eaf2c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:12.410529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bd623-c05e-4ad6-ad56-9051ceccfc14 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:12.412074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 553/1120] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:12 2022] GET /identity/v3/roles/98aaf5af7bb24b1596a9b01fd76eaf2c => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:12.418718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c33d4-0e7b-4fff-abee-727faefeb3e1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:12.519794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c33d4-0e7b-4fff-abee-727faefeb3e1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:12.531318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:12.555621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe4b55-63d1-4051-a070-bc9b4b918bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:12.556253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe4b55-63d1-4051-a070-bc9b4b918bfd None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:12.556527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe4b55-63d1-4051-a070-bc9b4b918bfd None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:12.556790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe4b55-63d1-4051-a070-bc9b4b918bfd None admin] PATH_INFO: `/v3/users/0aa73242cbdb4e01b165cb05a4510c5b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:12.557902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe4b55-63d1-4051-a070-bc9b4b918bfd None admin] RBAC: Authorizing `identity:get_user(user_id=0aa73242cbdb4e01b165cb05a4510c5b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:12.559373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe4b55-63d1-4051-a070-bc9b4b918bfd None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:12.560968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 568/1121] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:12 2022] GET /identity/v3/users/0aa73242cbdb4e01b165cb05a4510c5b => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:12.567181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bd623-c05e-4ad6-ad56-9051ceccfc14 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:12.674318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bd623-c05e-4ad6-ad56-9051ceccfc14 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:12.710611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea4da6-1d4a-44c7-9fac-11d7f2245620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:12.711338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea4da6-1d4a-44c7-9fac-11d7f2245620 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:12.711619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea4da6-1d4a-44c7-9fac-11d7f2245620 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:12.711911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea4da6-1d4a-44c7-9fac-11d7f2245620 None admin] PATH_INFO: `/v3/projects/01d68589735f44108dbf04bc2c9b762b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:12.713125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea4da6-1d4a-44c7-9fac-11d7f2245620 None admin] RBAC: Authorizing `identity:get_project(project_id=01d68589735f44108dbf04bc2c9b762b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:12.714624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea4da6-1d4a-44c7-9fac-11d7f2245620 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:12.716370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 554/1122] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:12 2022] GET /identity/v3/projects/01d68589735f44108dbf04bc2c9b762b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:12.722353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe4b55-63d1-4051-a070-bc9b4b918bfd None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:12.825268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe4b55-63d1-4051-a070-bc9b4b918bfd None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:12.859228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac7a52-c5df-4d3f-9bc4-279d9d9254d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:12.859878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7a52-c5df-4d3f-9bc4-279d9d9254d0 None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:12.860157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7a52-c5df-4d3f-9bc4-279d9d9254d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:12.860425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7a52-c5df-4d3f-9bc4-279d9d9254d0 None admin] PATH_INFO: `/v3/projects/01d68589735f44108dbf04bc2c9b762b/users/0aa73242cbdb4e01b165cb05a4510c5b/roles/98aaf5af7bb24b1596a9b01fd76eaf2c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:12.862643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac7a52-c5df-4d3f-9bc4-279d9d9254d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=01d68589735f44108dbf04bc2c9b762b, user_id=0aa73242cbdb4e01b165cb05a4510c5b, role_id=98aaf5af7bb24b1596a9b01fd76eaf2c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:12.864363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac7a52-c5df-4d3f-9bc4-279d9d9254d0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:12.950990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 569/1123] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:37:12 2022] PUT /identity/v3/projects/01d68589735f44108dbf04bc2c9b762b/users/0aa73242cbdb4e01b165cb05a4510c5b/roles/98aaf5af7bb24b1596a9b01fd76eaf2c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:37:14.872586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45c89-f3e2-4a97-b41c-c200a228f053 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:14.873125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45c89-f3e2-4a97-b41c-c200a228f053 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:14.873125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45c89-f3e2-4a97-b41c-c200a228f053 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:14.874447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 555/1124] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:14.879747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae212b1d-f8f2-4105-85c1-8a56d29fefe0 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:14.880182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae212b1d-f8f2-4105-85c1-8a56d29fefe0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:14.880300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae212b1d-f8f2-4105-85c1-8a56d29fefe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:14.898795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ae212b1d-f8f2-4105-85c1-8a56d29fefe0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:14.900010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ae212b1d-f8f2-4105-85c1-8a56d29fefe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:14.931020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:14.970761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 570/1125] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:15.245045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c45c89-f3e2-4a97-b41c-c200a228f053 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:15.256436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:15.346492 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c45c89-f3e2-4a97-b41c-c200a228f053 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:15.380071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b111c-e8b5-4fbf-b223-d9e729cbf4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:15.380694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b111c-e8b5-4fbf-b223-d9e729cbf4ca None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:15.381466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b111c-e8b5-4fbf-b223-d9e729cbf4ca None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:15.381732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b111c-e8b5-4fbf-b223-d9e729cbf4ca None admin] PATH_INFO: `/v3/roles/98aaf5af7bb24b1596a9b01fd76eaf2c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:15.383183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b111c-e8b5-4fbf-b223-d9e729cbf4ca None admin] RBAC: Authorizing `identity:get_role(role_id=98aaf5af7bb24b1596a9b01fd76eaf2c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:15.384559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b111c-e8b5-4fbf-b223-d9e729cbf4ca None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:15.385876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 556/1126] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:15 2022] GET /identity/v3/roles/98aaf5af7bb24b1596a9b01fd76eaf2c => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:15.391749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae212b1d-f8f2-4105-85c1-8a56d29fefe0 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:15.493984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae212b1d-f8f2-4105-85c1-8a56d29fefe0 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:15.527658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2363c-7526-4bb2-9071-bdbd05e33a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:15.528215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2363c-7526-4bb2-9071-bdbd05e33a83 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:15.528483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2363c-7526-4bb2-9071-bdbd05e33a83 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:15.528745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2363c-7526-4bb2-9071-bdbd05e33a83 None admin] PATH_INFO: `/v3/users/0aa73242cbdb4e01b165cb05a4510c5b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:15.529841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2363c-7526-4bb2-9071-bdbd05e33a83 None admin] RBAC: Authorizing `identity:get_user(user_id=0aa73242cbdb4e01b165cb05a4510c5b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:15.531287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2363c-7526-4bb2-9071-bdbd05e33a83 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:15.532842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 571/1127] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:15 2022] GET /identity/v3/users/0aa73242cbdb4e01b165cb05a4510c5b => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:15.538910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b111c-e8b5-4fbf-b223-d9e729cbf4ca None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:15.619144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:15.643737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b111c-e8b5-4fbf-b223-d9e729cbf4ca None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:15.679775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2013ca-92c1-4b2d-a182-9effc538000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:15.680371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2013ca-92c1-4b2d-a182-9effc538000c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:15.680644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2013ca-92c1-4b2d-a182-9effc538000c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:15.680909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2013ca-92c1-4b2d-a182-9effc538000c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:15.746279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2013ca-92c1-4b2d-a182-9effc538000c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:15.747744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2013ca-92c1-4b2d-a182-9effc538000c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:15.824218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:15.846684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 557/1128] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:37:15.853137 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2363c-7526-4bb2-9071-bdbd05e33a83 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:15.887643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:15.957088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2363c-7526-4bb2-9071-bdbd05e33a83 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:15.992970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723992e-c4d1-4f3b-88a0-63e907046f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:15.993587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723992e-c4d1-4f3b-88a0-63e907046f58 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:15.993856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723992e-c4d1-4f3b-88a0-63e907046f58 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:15.994118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723992e-c4d1-4f3b-88a0-63e907046f58 None admin] PATH_INFO: `/v3/projects/01d68589735f44108dbf04bc2c9b762b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:15.995303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723992e-c4d1-4f3b-88a0-63e907046f58 None admin] RBAC: Authorizing `identity:get_project(project_id=01d68589735f44108dbf04bc2c9b762b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:15.996720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723992e-c4d1-4f3b-88a0-63e907046f58 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:15.998368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 572/1129] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:15 2022] GET /identity/v3/projects/01d68589735f44108dbf04bc2c9b762b => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:16.004548 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2013ca-92c1-4b2d-a182-9effc538000c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:16.103390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2013ca-92c1-4b2d-a182-9effc538000c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:16.136887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45d02c-3141-41c3-80c5-85369a99075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:16.137487 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45d02c-3141-41c3-80c5-85369a99075f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:16.137765 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45d02c-3141-41c3-80c5-85369a99075f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:16.138041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45d02c-3141-41c3-80c5-85369a99075f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:16.138759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45d02c-3141-41c3-80c5-85369a99075f None admin] RBAC: Adding query filter params (role.id=98aaf5af7bb24b1596a9b01fd76eaf2c, scope.project.id=01d68589735f44108dbf04bc2c9b762b, user.id=0aa73242cbdb4e01b165cb05a4510c5b) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:16.139096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45d02c-3141-41c3-80c5-85369a99075f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:16.140522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45d02c-3141-41c3-80c5-85369a99075f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:16.146415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 558/1130] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:37:16 2022] GET /identity/v3/role_assignments?user.id=0aa73242cbdb4e01b165cb05a4510c5b&scope.project.id=01d68589735f44108dbf04bc2c9b762b&role.id=98aaf5af7bb24b1596a9b01fd76eaf2c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:18.166199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d505c-06ce-433f-83e5-f4f4aa9d6bf6 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:18.166757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d505c-06ce-433f-83e5-f4f4aa9d6bf6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:18.166757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d505c-06ce-433f-83e5-f4f4aa9d6bf6 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:18.168095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 573/1131] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:18.173342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4a0b-c170-4983-bd54-6034310db2d6 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:18.173752 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4a0b-c170-4983-bd54-6034310db2d6 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:18.173924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4a0b-c170-4983-bd54-6034310db2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:18.192595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-b70e4a0b-c170-4983-bd54-6034310db2d6 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:18.194350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b70e4a0b-c170-4983-bd54-6034310db2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:18.230210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:18.263659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 559/1132] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:18.536578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d505c-06ce-433f-83e5-f4f4aa9d6bf6 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:18.645808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d505c-06ce-433f-83e5-f4f4aa9d6bf6 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:18.648327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:18.681954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88c682-ba5d-497e-bb3d-7a981257d2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:18.682582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88c682-ba5d-497e-bb3d-7a981257d2ff None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:18.682889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88c682-ba5d-497e-bb3d-7a981257d2ff None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:18.683193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88c682-ba5d-497e-bb3d-7a981257d2ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:18.684321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88c682-ba5d-497e-bb3d-7a981257d2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:18.685757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88c682-ba5d-497e-bb3d-7a981257d2ff None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:18.687716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 574/1133] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:37:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:18.694871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e4a0b-c170-4983-bd54-6034310db2d6 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:18.799469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e4a0b-c170-4983-bd54-6034310db2d6 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:18.836451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543684f-9131-4124-871f-1d3010ffe31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:18.837048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543684f-9131-4124-871f-1d3010ffe31b None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:18.837318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543684f-9131-4124-871f-1d3010ffe31b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:18.837584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543684f-9131-4124-871f-1d3010ffe31b None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:18.838216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543684f-9131-4124-871f-1d3010ffe31b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:18.839670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543684f-9131-4124-871f-1d3010ffe31b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:18.940714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 560/1134] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:37:18 2022] POST /identity/v3/projects => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:37:20.900565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a97a0-8c66-4e29-bd6a-390cbcffcefd None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:20.901102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a97a0-8c66-4e29-bd6a-390cbcffcefd None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:20.901102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a97a0-8c66-4e29-bd6a-390cbcffcefd None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:20.902412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 575/1135] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:20.907895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eaabe-2186-4a92-ab60-169a320c8571 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:20.908302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eaabe-2186-4a92-ab60-169a320c8571 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:20.908430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17eaabe-2186-4a92-ab60-169a320c8571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:20.927341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c17eaabe-2186-4a92-ab60-169a320c8571 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:20.928583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c17eaabe-2186-4a92-ab60-169a320c8571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:20.945095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:20.997213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 561/1136] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:21.273068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a97a0-8c66-4e29-bd6a-390cbcffcefd None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:21.374158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a97a0-8c66-4e29-bd6a-390cbcffcefd None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:21.408344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a0a95-d616-445f-bf4f-6c6a05bff6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:21.408958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a0a95-d616-445f-bf4f-6c6a05bff6b2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:21.409230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a0a95-d616-445f-bf4f-6c6a05bff6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:21.409492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a0a95-d616-445f-bf4f-6c6a05bff6b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:21.410587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a0a95-d616-445f-bf4f-6c6a05bff6b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:21.412057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a0a95-d616-445f-bf4f-6c6a05bff6b2 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:21.413600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 576/1137] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:37:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:21.419842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17eaabe-2186-4a92-ab60-169a320c8571 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:21.526203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17eaabe-2186-4a92-ab60-169a320c8571 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:21.560739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4b78d-83d5-4863-ab7f-da7e1e3b3609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:21.561434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4b78d-83d5-4863-ab7f-da7e1e3b3609 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:21.561715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4b78d-83d5-4863-ab7f-da7e1e3b3609 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:21.561992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4b78d-83d5-4863-ab7f-da7e1e3b3609 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:21.562651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4b78d-83d5-4863-ab7f-da7e1e3b3609 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:21.564137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4b78d-83d5-4863-ab7f-da7e1e3b3609 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:21.674526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 562/1138] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:37:21 2022] POST /identity/v3/users => generated 286 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:37:23.722545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1859c-fbbc-4657-9306-4713a47ead70 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:23.723131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1859c-fbbc-4657-9306-4713a47ead70 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:23.723131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1859c-fbbc-4657-9306-4713a47ead70 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:23.725205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 577/1139] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:23.731032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf43b63-11b7-41b9-bdaa-98e0fb09b5b0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:23.731465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf43b63-11b7-41b9-bdaa-98e0fb09b5b0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:23.731583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf43b63-11b7-41b9-bdaa-98e0fb09b5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:23.750219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-abf43b63-11b7-41b9-bdaa-98e0fb09b5b0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:23.751445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-abf43b63-11b7-41b9-bdaa-98e0fb09b5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:23.753401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:23.819641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 563/1140] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:24.096272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1859c-fbbc-4657-9306-4713a47ead70 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:24.140872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:24.200692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1859c-fbbc-4657-9306-4713a47ead70 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:24.235416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31fc2c-4cc1-4f92-b6bc-d0f06474a0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:24.236040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31fc2c-4cc1-4f92-b6bc-d0f06474a0f6 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:24.236315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31fc2c-4cc1-4f92-b6bc-d0f06474a0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:24.236583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31fc2c-4cc1-4f92-b6bc-d0f06474a0f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:24.247714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31fc2c-4cc1-4f92-b6bc-d0f06474a0f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-4f31fc2c-4cc1-4f92-b6bc-d0f06474a0f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:37:24.252320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:24.253460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:37:24.254745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 578/1141] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:37:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:24.259490 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf43b63-11b7-41b9-bdaa-98e0fb09b5b0 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:24.366287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf43b63-11b7-41b9-bdaa-98e0fb09b5b0 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:24.399366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:24.399968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:24.400243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:24.400511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:24.401171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:24.401472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:24.402827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:24.407816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:24.411133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 564/1142] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:37:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:24.417475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31fc2c-4cc1-4f92-b6bc-d0f06474a0f6 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:24.614241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:24.617248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31fc2c-4cc1-4f92-b6bc-d0f06474a0f6 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:24.651697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc947697-c4ec-45ab-a299-38fc1e873038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:24.652316 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947697-c4ec-45ab-a299-38fc1e873038 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:24.652588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947697-c4ec-45ab-a299-38fc1e873038 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:24.652850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947697-c4ec-45ab-a299-38fc1e873038 None admin] PATH_INFO: `/v3/users/0de3c08ea7384cfd98fe39959eec4a4f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:24.666671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc947697-c4ec-45ab-a299-38fc1e873038 None admin] RBAC: Authorizing `identity:get_user(user_id=0de3c08ea7384cfd98fe39959eec4a4f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:24.668132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc947697-c4ec-45ab-a299-38fc1e873038 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:24.669776 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 579/1143] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:24 2022] GET /identity/v3/users/0de3c08ea7384cfd98fe39959eec4a4f => generated 286 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:24.675890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:24.777965 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394c247-acb9-4d8c-9ea3-b20afdc581ca None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:24.812115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1759a-05de-477e-a364-65e8a9a24a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:24.812708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1759a-05de-477e-a364-65e8a9a24a4e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:24.812980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1759a-05de-477e-a364-65e8a9a24a4e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:24.813245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1759a-05de-477e-a364-65e8a9a24a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:24.877662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:24.879543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1759a-05de-477e-a364-65e8a9a24a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:24.881495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1759a-05de-477e-a364-65e8a9a24a4e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:24.978474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 565/1144] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:37:24.984462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc947697-c4ec-45ab-a299-38fc1e873038 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:25.086073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc947697-c4ec-45ab-a299-38fc1e873038 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:25.119624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5b75-698d-4b9f-9f6d-23d80ebcd773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:25.120246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5b75-698d-4b9f-9f6d-23d80ebcd773 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:25.120522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5b75-698d-4b9f-9f6d-23d80ebcd773 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:25.120787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5b75-698d-4b9f-9f6d-23d80ebcd773 None admin] PATH_INFO: `/v3/projects/3bc190005e0e415cbcfde1d44d85ff32` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:25.121891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b5b75-698d-4b9f-9f6d-23d80ebcd773 None admin] RBAC: Authorizing `identity:get_project(project_id=3bc190005e0e415cbcfde1d44d85ff32)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:25.123346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b5b75-698d-4b9f-9f6d-23d80ebcd773 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:25.124960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 580/1145] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:24 2022] GET /identity/v3/projects/3bc190005e0e415cbcfde1d44d85ff32 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:25.131663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1759a-05de-477e-a364-65e8a9a24a4e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:25.227495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:25.230404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1759a-05de-477e-a364-65e8a9a24a4e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:25.265529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a6c59-db17-45c2-86f1-361d3f2667ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:25.266126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6c59-db17-45c2-86f1-361d3f2667ab None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:25.266404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6c59-db17-45c2-86f1-361d3f2667ab None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:25.266675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6c59-db17-45c2-86f1-361d3f2667ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:25.267428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a6c59-db17-45c2-86f1-361d3f2667ab None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=3bc190005e0e415cbcfde1d44d85ff32, user.id=0de3c08ea7384cfd98fe39959eec4a4f) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:25.267728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a6c59-db17-45c2-86f1-361d3f2667ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:25.269193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a6c59-db17-45c2-86f1-361d3f2667ab None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:25.275364 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 566/1146] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:37:25 2022] GET /identity/v3/role_assignments?user.id=0de3c08ea7384cfd98fe39959eec4a4f&scope.project.id=3bc190005e0e415cbcfde1d44d85ff32&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:27.149294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51754e-04e4-416d-9e87-246666e5e728 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:27.149845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51754e-04e4-416d-9e87-246666e5e728 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:27.149845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51754e-04e4-416d-9e87-246666e5e728 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:27.151206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 581/1147] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:27.157038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0172a9-dffa-49f4-a8e1-2b8382ac1194 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:27.157465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0172a9-dffa-49f4-a8e1-2b8382ac1194 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:27.157564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0172a9-dffa-49f4-a8e1-2b8382ac1194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:27.178464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ab0172a9-dffa-49f4-a8e1-2b8382ac1194 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:27.179715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ab0172a9-dffa-49f4-a8e1-2b8382ac1194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:27.250891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 567/1148] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:27.554385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51754e-04e4-416d-9e87-246666e5e728 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:27.585265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:27.673821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51754e-04e4-416d-9e87-246666e5e728 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:27.718030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37184c-ee10-468d-8732-4bbe3af1b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:27.718030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37184c-ee10-468d-8732-4bbe3af1b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:27.718030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37184c-ee10-468d-8732-4bbe3af1b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:27.718030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37184c-ee10-468d-8732-4bbe3af1b2f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:27.734340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37184c-ee10-468d-8732-4bbe3af1b2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-7f37184c-ee10-468d-8732-4bbe3af1b2f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:37:27.742584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:27.744089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:37:27.744949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 582/1149] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:37:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:27.752110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0172a9-dffa-49f4-a8e1-2b8382ac1194 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:27.857205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:27.866337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0172a9-dffa-49f4-a8e1-2b8382ac1194 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:27.905486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:27.906138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:27.906417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:27.906693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:27.907393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:27.907693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:27.909054 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:27.917823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 568/1150] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:27.928160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37184c-ee10-468d-8732-4bbe3af1b2f2 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:28.035333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:28.051045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37184c-ee10-468d-8732-4bbe3af1b2f2 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:28.095747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac795cf-b446-4e57-83db-0614f58ace1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:28.096846 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac795cf-b446-4e57-83db-0614f58ace1c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:28.097321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac795cf-b446-4e57-83db-0614f58ace1c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:28.097599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac795cf-b446-4e57-83db-0614f58ace1c None admin] PATH_INFO: `/v3/users/0de3c08ea7384cfd98fe39959eec4a4f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:28.098742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac795cf-b446-4e57-83db-0614f58ace1c None admin] RBAC: Authorizing `identity:get_user(user_id=0de3c08ea7384cfd98fe39959eec4a4f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:28.101170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac795cf-b446-4e57-83db-0614f58ace1c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:28.102753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 583/1151] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:27 2022] GET /identity/v3/users/0de3c08ea7384cfd98fe39959eec4a4f => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:28.108780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:28.210956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd084f-c6fe-45fa-9ae9-338c98b375bd None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:28.246797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b02eb-26a1-49ce-8b2c-6601b05fd48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:28.247452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b02eb-26a1-49ce-8b2c-6601b05fd48b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:28.247717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b02eb-26a1-49ce-8b2c-6601b05fd48b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:28.247982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b02eb-26a1-49ce-8b2c-6601b05fd48b None admin] PATH_INFO: `/v3/projects/3bc190005e0e415cbcfde1d44d85ff32` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:28.249087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b02eb-26a1-49ce-8b2c-6601b05fd48b None admin] RBAC: Authorizing `identity:get_project(project_id=3bc190005e0e415cbcfde1d44d85ff32)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:28.250499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b02eb-26a1-49ce-8b2c-6601b05fd48b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:28.252157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 569/1152] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:28 2022] GET /identity/v3/projects/3bc190005e0e415cbcfde1d44d85ff32 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:28.258349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac795cf-b446-4e57-83db-0614f58ace1c None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:28.361199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac795cf-b446-4e57-83db-0614f58ace1c None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:28.395615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157442b-a9dc-4a41-82ee-d0307941a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:28.396832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157442b-a9dc-4a41-82ee-d0307941a13d None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:28.397100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157442b-a9dc-4a41-82ee-d0307941a13d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:28.397590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157442b-a9dc-4a41-82ee-d0307941a13d None admin] PATH_INFO: `/v3/projects/3bc190005e0e415cbcfde1d44d85ff32/users/0de3c08ea7384cfd98fe39959eec4a4f/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:28.399816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157442b-a9dc-4a41-82ee-d0307941a13d None admin] RBAC: Authorizing `identity:create_grant(project_id=3bc190005e0e415cbcfde1d44d85ff32, user_id=0de3c08ea7384cfd98fe39959eec4a4f, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:28.401293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157442b-a9dc-4a41-82ee-d0307941a13d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:28.509447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 584/1153] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:37:28 2022] PUT /identity/v3/projects/3bc190005e0e415cbcfde1d44d85ff32/users/0de3c08ea7384cfd98fe39959eec4a4f/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:37:30.439507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4160853-f980-4988-98a7-e79415de837d None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:30.440054 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4160853-f980-4988-98a7-e79415de837d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:30.440054 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4160853-f980-4988-98a7-e79415de837d None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:30.441486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 570/1154] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:30.447773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e2981-66fa-4a3d-888d-508b1fb729a7 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:30.448270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e2981-66fa-4a3d-888d-508b1fb729a7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:30.448380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e2981-66fa-4a3d-888d-508b1fb729a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:30.466206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4a5e2981-66fa-4a3d-888d-508b1fb729a7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:30.467420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4a5e2981-66fa-4a3d-888d-508b1fb729a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:30.478003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:30.538035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 585/1155] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:30.815580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4160853-f980-4988-98a7-e79415de837d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:30.841201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:30.921091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4160853-f980-4988-98a7-e79415de837d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:30.957585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81191c8c-5ae2-45b3-812f-e028f8b15f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:30.958199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81191c8c-5ae2-45b3-812f-e028f8b15f8c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:30.958513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81191c8c-5ae2-45b3-812f-e028f8b15f8c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:30.958783 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81191c8c-5ae2-45b3-812f-e028f8b15f8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:30.970108 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81191c8c-5ae2-45b3-812f-e028f8b15f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-81191c8c-5ae2-45b3-812f-e028f8b15f8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:37:30.975464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:30.976663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:37:30.977578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 571/1156] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:37:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:30.983160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e2981-66fa-4a3d-888d-508b1fb729a7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:31.084820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e2981-66fa-4a3d-888d-508b1fb729a7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:31.120357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:31.120961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:31.121241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:31.121506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:31.122168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:31.122466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:31.123842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:31.131942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 586/1157] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:37:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:37:31.137888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81191c8c-5ae2-45b3-812f-e028f8b15f8c None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:31.223243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:31.239564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81191c8c-5ae2-45b3-812f-e028f8b15f8c None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:31.273046 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b89f4-21f6-4538-ac7f-5a1e5b2a7e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:31.273665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b89f4-21f6-4538-ac7f-5a1e5b2a7e76 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:31.273943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b89f4-21f6-4538-ac7f-5a1e5b2a7e76 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:31.274213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b89f4-21f6-4538-ac7f-5a1e5b2a7e76 None admin] PATH_INFO: `/v3/users/0de3c08ea7384cfd98fe39959eec4a4f` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:31.275355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b89f4-21f6-4538-ac7f-5a1e5b2a7e76 None admin] RBAC: Authorizing `identity:get_user(user_id=0de3c08ea7384cfd98fe39959eec4a4f)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:31.276779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b89f4-21f6-4538-ac7f-5a1e5b2a7e76 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:31.278350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 572/1158] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:31 2022] GET /identity/v3/users/0de3c08ea7384cfd98fe39959eec4a4f => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:31.284425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:31.294727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:31.392164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e388025-37cb-4d2b-a95b-39aaa3dbe109 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:31.431264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ceb1d-fc3a-4a30-bcb4-43f9853953fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:31.431887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ceb1d-fc3a-4a30-bcb4-43f9853953fc None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:31.432155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ceb1d-fc3a-4a30-bcb4-43f9853953fc None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:31.432421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ceb1d-fc3a-4a30-bcb4-43f9853953fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:31.504935 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ceb1d-fc3a-4a30-bcb4-43f9853953fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:31.506583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ceb1d-fc3a-4a30-bcb4-43f9853953fc None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:31.514886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:31.608286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 587/1159] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:37:31.614754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b89f4-21f6-4538-ac7f-5a1e5b2a7e76 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:31.717173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b89f4-21f6-4538-ac7f-5a1e5b2a7e76 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:31.750774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a9dee-3e31-44d2-9993-9b6382ef9c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:31.751429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9dee-3e31-44d2-9993-9b6382ef9c06 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:31.751704 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9dee-3e31-44d2-9993-9b6382ef9c06 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:31.751975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9dee-3e31-44d2-9993-9b6382ef9c06 None admin] PATH_INFO: `/v3/projects/3bc190005e0e415cbcfde1d44d85ff32` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:31.753105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a9dee-3e31-44d2-9993-9b6382ef9c06 None admin] RBAC: Authorizing `identity:get_project(project_id=3bc190005e0e415cbcfde1d44d85ff32)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:31.754524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a9dee-3e31-44d2-9993-9b6382ef9c06 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:31.756912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 573/1160] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:31 2022] GET /identity/v3/projects/3bc190005e0e415cbcfde1d44d85ff32 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:31.763167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ceb1d-fc3a-4a30-bcb4-43f9853953fc None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:31.877336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ceb1d-fc3a-4a30-bcb4-43f9853953fc None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:31.885356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:31.912541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48530cae-2014-4187-85c3-6a6b41d0f751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:31.913142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530cae-2014-4187-85c3-6a6b41d0f751 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:31.913416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530cae-2014-4187-85c3-6a6b41d0f751 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:31.913685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530cae-2014-4187-85c3-6a6b41d0f751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:31.914400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48530cae-2014-4187-85c3-6a6b41d0f751 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=3bc190005e0e415cbcfde1d44d85ff32, user.id=0de3c08ea7384cfd98fe39959eec4a4f) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:31.914703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48530cae-2014-4187-85c3-6a6b41d0f751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:31.916162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48530cae-2014-4187-85c3-6a6b41d0f751 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:31.921835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 588/1161] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:37:31 2022] GET /identity/v3/role_assignments?user.id=0de3c08ea7384cfd98fe39959eec4a4f&scope.project.id=3bc190005e0e415cbcfde1d44d85ff32&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:33.932183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567791-24de-4453-96f5-89c54a3f3bd5 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:33.932743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567791-24de-4453-96f5-89c54a3f3bd5 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:33.932743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43567791-24de-4453-96f5-89c54a3f3bd5 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:33.934119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 574/1162] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:33.940251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decded3-ec63-407c-97f3-a49802c598bc None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:33.940755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decded3-ec63-407c-97f3-a49802c598bc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:33.940848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decded3-ec63-407c-97f3-a49802c598bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:33.958905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5decded3-ec63-407c-97f3-a49802c598bc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:33.960141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5decded3-ec63-407c-97f3-a49802c598bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:34.032156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 589/1163] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:34.324868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43567791-24de-4453-96f5-89c54a3f3bd5 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:34.355860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:34.447512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43567791-24de-4453-96f5-89c54a3f3bd5 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:34.489579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a080a34-d3fb-4af6-a932-695339d93540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:34.490574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a080a34-d3fb-4af6-a932-695339d93540 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:34.491072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a080a34-d3fb-4af6-a932-695339d93540 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:34.491400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a080a34-d3fb-4af6-a932-695339d93540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:34.560735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a080a34-d3fb-4af6-a932-695339d93540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:34.562230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a080a34-d3fb-4af6-a932-695339d93540 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:34.584674 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:34.663228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 575/1164] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:37:34.669281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decded3-ec63-407c-97f3-a49802c598bc None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:34.780282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:34.783243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decded3-ec63-407c-97f3-a49802c598bc None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:34.817775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a3293-7155-40f7-b3ec-4a1128473595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:34.818509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a3293-7155-40f7-b3ec-4a1128473595 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:34.818783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a3293-7155-40f7-b3ec-4a1128473595 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:34.819089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a3293-7155-40f7-b3ec-4a1128473595 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:34.825667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a3293-7155-40f7-b3ec-4a1128473595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:34.827133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a3293-7155-40f7-b3ec-4a1128473595 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8d5a3293-7155-40f7-b3ec-4a1128473595 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:34.838432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:34.839644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 15:37:34.840654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:37:34.843069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 590/1165] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:37:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:34.845525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a080a34-d3fb-4af6-a932-695339d93540 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:34.950310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a080a34-d3fb-4af6-a932-695339d93540 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:34.970288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:34.984311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ae0d8-9a12-485f-ab38-a9192da6319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:34.991087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ae0d8-9a12-485f-ab38-a9192da6319e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:34.991087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ae0d8-9a12-485f-ab38-a9192da6319e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:34.991087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ae0d8-9a12-485f-ab38-a9192da6319e None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:34.991087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ae0d8-9a12-485f-ab38-a9192da6319e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:34.991087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ae0d8-9a12-485f-ab38-a9192da6319e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:34.991087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ae0d8-9a12-485f-ab38-a9192da6319e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:34.995367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 576/1166] 10.222.0.94 () {56 vars in 1115 bytes} [Fri Aug 26 15:37:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:37.213999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac60589-fa6b-4754-9375-3a53d3df0246 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:37.214750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac60589-fa6b-4754-9375-3a53d3df0246 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:37.214750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac60589-fa6b-4754-9375-3a53d3df0246 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:37.215905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 591/1167] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:37.221428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a664-9784-40f8-a36e-691730601266 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:37.221901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a664-9784-40f8-a36e-691730601266 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:37.222029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a664-9784-40f8-a36e-691730601266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:37.241397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-24c1a664-9784-40f8-a36e-691730601266 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:37.242587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-24c1a664-9784-40f8-a36e-691730601266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:37.311660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 577/1168] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:37.591214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac60589-fa6b-4754-9375-3a53d3df0246 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:37.699196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac60589-fa6b-4754-9375-3a53d3df0246 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:37.735428 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe2d63-b2e7-42ce-a069-a5927e77744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:37.736040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe2d63-b2e7-42ce-a069-a5927e77744e None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:37.736312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe2d63-b2e7-42ce-a069-a5927e77744e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:37.736578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe2d63-b2e7-42ce-a069-a5927e77744e None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:37.737248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe2d63-b2e7-42ce-a069-a5927e77744e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:37.738881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe2d63-b2e7-42ce-a069-a5927e77744e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:37.839071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 592/1169] 10.222.0.94 () {60 vars in 1138 bytes} [Fri Aug 26 15:37:37 2022] POST /identity/v3/domains => generated 267 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:37:39.812579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3faf-e639-443d-aec7-927d573de7ae None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:39.813104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3faf-e639-443d-aec7-927d573de7ae None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:39.813104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3faf-e639-443d-aec7-927d573de7ae None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:39.814442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 578/1170] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:39.820348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e600e4d-3a63-477f-8b81-27fa389ec20b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:39.820752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e600e4d-3a63-477f-8b81-27fa389ec20b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:39.820891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e600e4d-3a63-477f-8b81-27fa389ec20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:39.846232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6e600e4d-3a63-477f-8b81-27fa389ec20b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:39.847379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6e600e4d-3a63-477f-8b81-27fa389ec20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:39.861978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:39.916488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 593/1171] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:40.205229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a3faf-e639-443d-aec7-927d573de7ae None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:40.309735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a3faf-e639-443d-aec7-927d573de7ae None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:40.324808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:40.345036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb99c5-d3ac-49a3-a1eb-78f2e46bd54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:40.345668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb99c5-d3ac-49a3-a1eb-78f2e46bd54c None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:40.345938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb99c5-d3ac-49a3-a1eb-78f2e46bd54c None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:40.346204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb99c5-d3ac-49a3-a1eb-78f2e46bd54c None admin] PATH_INFO: `/v3/domains/2d149200bdcd4bb58945e962bbc897be` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:40.357086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb99c5-d3ac-49a3-a1eb-78f2e46bd54c None admin] RBAC: Authorizing `identity:get_domain(domain_id=2d149200bdcd4bb58945e962bbc897be)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:40.358631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb99c5-d3ac-49a3-a1eb-78f2e46bd54c None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:40.360318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 579/1172] 10.222.0.94 () {56 vars in 1183 bytes} [Fri Aug 26 15:37:40 2022] GET /identity/v3/domains/2d149200bdcd4bb58945e962bbc897be => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:40.367134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e600e4d-3a63-477f-8b81-27fa389ec20b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:40.477455 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e600e4d-3a63-477f-8b81-27fa389ec20b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:40.512830 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b425284-9885-4e73-8f1e-edb930dd6ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:40.513450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425284-9885-4e73-8f1e-edb930dd6ed1 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:40.513726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425284-9885-4e73-8f1e-edb930dd6ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:40.513994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425284-9885-4e73-8f1e-edb930dd6ed1 None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:40.514675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b425284-9885-4e73-8f1e-edb930dd6ed1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:40.516149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b425284-9885-4e73-8f1e-edb930dd6ed1 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:40.606662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 594/1173] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:37:40 2022] POST /identity/v3/projects => generated 361 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:37:42.579318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913e090-3e66-4d35-a6df-9d064e281723 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:42.579894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913e090-3e66-4d35-a6df-9d064e281723 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:42.579894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913e090-3e66-4d35-a6df-9d064e281723 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:42.581191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 580/1174] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:42.587644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049a77-d906-4ce1-b2b5-1ad180233f7f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:42.587644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049a77-d906-4ce1-b2b5-1ad180233f7f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:42.587644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049a77-d906-4ce1-b2b5-1ad180233f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:42.593921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:42.606340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ed049a77-d906-4ce1-b2b5-1ad180233f7f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:42.608736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ed049a77-d906-4ce1-b2b5-1ad180233f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:42.677458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 595/1175] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:42.955383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913e090-3e66-4d35-a6df-9d064e281723 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:43.059398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913e090-3e66-4d35-a6df-9d064e281723 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:43.093339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51929a-ebcf-4ddb-81f5-6bac7869f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:43.093973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51929a-ebcf-4ddb-81f5-6bac7869f250 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:43.094248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51929a-ebcf-4ddb-81f5-6bac7869f250 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:43.094517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51929a-ebcf-4ddb-81f5-6bac7869f250 None admin] PATH_INFO: `/v3/domains/2d149200bdcd4bb58945e962bbc897be` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:43.095661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51929a-ebcf-4ddb-81f5-6bac7869f250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2d149200bdcd4bb58945e962bbc897be)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:43.097090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51929a-ebcf-4ddb-81f5-6bac7869f250 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:43.099340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 581/1176] 10.222.0.94 () {56 vars in 1183 bytes} [Fri Aug 26 15:37:42 2022] GET /identity/v3/domains/2d149200bdcd4bb58945e962bbc897be => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:43.105228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed049a77-d906-4ce1-b2b5-1ad180233f7f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:43.207024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed049a77-d906-4ce1-b2b5-1ad180233f7f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:43.238731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:43.244204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44121c1-f62a-4963-8a3e-8b7b98e2bcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:43.244812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44121c1-f62a-4963-8a3e-8b7b98e2bcb0 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:43.245083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44121c1-f62a-4963-8a3e-8b7b98e2bcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:43.245347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44121c1-f62a-4963-8a3e-8b7b98e2bcb0 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:43.245991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44121c1-f62a-4963-8a3e-8b7b98e2bcb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:43.247592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44121c1-f62a-4963-8a3e-8b7b98e2bcb0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:43.363408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 596/1177] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:37:43 2022] POST /identity/v3/users => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:37:45.345423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00889e-a423-4035-be25-bf75570500cf None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:45.345990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00889e-a423-4035-be25-bf75570500cf None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:45.345990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00889e-a423-4035-be25-bf75570500cf None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:45.347300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 582/1178] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:45.353176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5955c-4012-4909-af66-8064bcafc027 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:45.353739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5955c-4012-4909-af66-8064bcafc027 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:45.354020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5955c-4012-4909-af66-8064bcafc027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:45.372367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-90c5955c-4012-4909-af66-8064bcafc027 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:45.373570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-90c5955c-4012-4909-af66-8064bcafc027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:45.443777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 597/1179] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:45.726048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00889e-a423-4035-be25-bf75570500cf None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:45.772717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:45.831941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00889e-a423-4035-be25-bf75570500cf None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:45.865738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0031f2b-2498-4891-9b14-371a23cee53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:45.866362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0031f2b-2498-4891-9b14-371a23cee53f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:45.866632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0031f2b-2498-4891-9b14-371a23cee53f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:45.866895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0031f2b-2498-4891-9b14-371a23cee53f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:45.877531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0031f2b-2498-4891-9b14-371a23cee53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-a0031f2b-2498-4891-9b14-371a23cee53f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:37:45.882673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:45.883873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:37:45.885143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 583/1180] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:37:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:45.889862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5955c-4012-4909-af66-8064bcafc027 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:45.990985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5955c-4012-4909-af66-8064bcafc027 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:46.008892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:46.026335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:46.026961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:46.027242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:46.027512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:46.028179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:46.028502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:46.029877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:46.038230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 598/1181] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:37:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:46.044018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0031f2b-2498-4891-9b14-371a23cee53f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:46.149034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0031f2b-2498-4891-9b14-371a23cee53f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:46.150595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:46.183231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b873db5-6058-4f4c-b8be-12a463282b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:46.183862 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b873db5-6058-4f4c-b8be-12a463282b86 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:46.184139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b873db5-6058-4f4c-b8be-12a463282b86 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:46.184409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b873db5-6058-4f4c-b8be-12a463282b86 None admin] PATH_INFO: `/v3/users/bbfe89da98284345b0237770cd2ac585` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:46.198781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b873db5-6058-4f4c-b8be-12a463282b86 None admin] RBAC: Authorizing `identity:get_user(user_id=bbfe89da98284345b0237770cd2ac585)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:46.200252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b873db5-6058-4f4c-b8be-12a463282b86 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:46.201902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 584/1182] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:46 2022] GET /identity/v3/users/bbfe89da98284345b0237770cd2ac585 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:46.208062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:46.312489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135646a-7d85-4476-9cd2-846cca47b512 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:46.347025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663702b0-351d-4072-b26d-3bfbc4a4a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:46.347627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663702b0-351d-4072-b26d-3bfbc4a4a04d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:46.347904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663702b0-351d-4072-b26d-3bfbc4a4a04d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:46.348172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663702b0-351d-4072-b26d-3bfbc4a4a04d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:46.397833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:46.419328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663702b0-351d-4072-b26d-3bfbc4a4a04d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:46.420807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663702b0-351d-4072-b26d-3bfbc4a4a04d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:46.523800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 599/1183] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:37:46.529949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b873db5-6058-4f4c-b8be-12a463282b86 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:46.636906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b873db5-6058-4f4c-b8be-12a463282b86 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:46.672962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63a0b4-9539-47e5-8d43-95ffef7c2a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:46.673598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63a0b4-9539-47e5-8d43-95ffef7c2a98 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:46.673875 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63a0b4-9539-47e5-8d43-95ffef7c2a98 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:46.674146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63a0b4-9539-47e5-8d43-95ffef7c2a98 None admin] PATH_INFO: `/v3/projects/64e3a120a396491882f930e29a8107fa` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:46.675413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63a0b4-9539-47e5-8d43-95ffef7c2a98 None admin] RBAC: Authorizing `identity:get_project(project_id=64e3a120a396491882f930e29a8107fa)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:46.677561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63a0b4-9539-47e5-8d43-95ffef7c2a98 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:46.679235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 585/1184] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:46 2022] GET /identity/v3/projects/64e3a120a396491882f930e29a8107fa => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:46.685737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663702b0-351d-4072-b26d-3bfbc4a4a04d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:46.768024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:46.788271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663702b0-351d-4072-b26d-3bfbc4a4a04d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:46.823714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d23ba-b7f9-467a-ac21-f7987a16abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:46.824313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d23ba-b7f9-467a-ac21-f7987a16abe9 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:46.824584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d23ba-b7f9-467a-ac21-f7987a16abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:46.824891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d23ba-b7f9-467a-ac21-f7987a16abe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:46.825552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d23ba-b7f9-467a-ac21-f7987a16abe9 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=64e3a120a396491882f930e29a8107fa, user.id=bbfe89da98284345b0237770cd2ac585) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:46.825852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d23ba-b7f9-467a-ac21-f7987a16abe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:46.827277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d23ba-b7f9-467a-ac21-f7987a16abe9 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:46.832694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 600/1185] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:37:46 2022] GET /identity/v3/role_assignments?user.id=bbfe89da98284345b0237770cd2ac585&scope.project.id=64e3a120a396491882f930e29a8107fa&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:48.746841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff11692-ad5f-4595-8f05-82afa0f78312 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:48.747424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff11692-ad5f-4595-8f05-82afa0f78312 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:48.747424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff11692-ad5f-4595-8f05-82afa0f78312 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:48.748841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 586/1186] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:48.754145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227c3b-0b31-4a99-b95b-c4df279dd5a7 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:48.755405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227c3b-0b31-4a99-b95b-c4df279dd5a7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:48.755405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227c3b-0b31-4a99-b95b-c4df279dd5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:48.773498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4c227c3b-0b31-4a99-b95b-c4df279dd5a7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:48.774726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4c227c3b-0b31-4a99-b95b-c4df279dd5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:48.843499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 601/1187] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:49.122312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff11692-ad5f-4595-8f05-82afa0f78312 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:49.152822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:49.227454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff11692-ad5f-4595-8f05-82afa0f78312 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:49.262604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5362fe6-d5e5-41db-9b92-985d54ea21f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:49.263265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362fe6-d5e5-41db-9b92-985d54ea21f0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:49.263535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362fe6-d5e5-41db-9b92-985d54ea21f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:49.263800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362fe6-d5e5-41db-9b92-985d54ea21f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:49.276301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5362fe6-d5e5-41db-9b92-985d54ea21f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-b5362fe6-d5e5-41db-9b92-985d54ea21f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:37:49.281012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:49.282132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:37:49.283153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 587/1188] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:37:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:49.289337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c227c3b-0b31-4a99-b95b-c4df279dd5a7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:49.369312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:49.395595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c227c3b-0b31-4a99-b95b-c4df279dd5a7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:49.432577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:49.433186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:49.433487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:49.433731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:49.434460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:49.434752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:49.436155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:49.445682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 602/1189] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:37:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:49.451721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5362fe6-d5e5-41db-9b92-985d54ea21f0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:49.546984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:49.561563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5362fe6-d5e5-41db-9b92-985d54ea21f0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:49.599817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0e30f-c294-4ac3-9118-63ebb1ad6c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:49.600453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0e30f-c294-4ac3-9118-63ebb1ad6c51 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:49.600728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0e30f-c294-4ac3-9118-63ebb1ad6c51 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:49.600996 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0e30f-c294-4ac3-9118-63ebb1ad6c51 None admin] PATH_INFO: `/v3/users/bbfe89da98284345b0237770cd2ac585` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:49.602157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0e30f-c294-4ac3-9118-63ebb1ad6c51 None admin] RBAC: Authorizing `identity:get_user(user_id=bbfe89da98284345b0237770cd2ac585)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:49.603610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0e30f-c294-4ac3-9118-63ebb1ad6c51 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:49.605202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 588/1190] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:49 2022] GET /identity/v3/users/bbfe89da98284345b0237770cd2ac585 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:49.611078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:49.713496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9369b2-befa-4aa2-a279-2de73bf213f8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:49.748470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97b9c2-13e8-43e0-84e9-b0f97ef334a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:49.749095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97b9c2-13e8-43e0-84e9-b0f97ef334a0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:49.749370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97b9c2-13e8-43e0-84e9-b0f97ef334a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:49.749641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97b9c2-13e8-43e0-84e9-b0f97ef334a0 None admin] PATH_INFO: `/v3/projects/64e3a120a396491882f930e29a8107fa` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:49.750759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97b9c2-13e8-43e0-84e9-b0f97ef334a0 None admin] RBAC: Authorizing `identity:get_project(project_id=64e3a120a396491882f930e29a8107fa)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:49.752219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97b9c2-13e8-43e0-84e9-b0f97ef334a0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:49.753852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 603/1191] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:49 2022] GET /identity/v3/projects/64e3a120a396491882f930e29a8107fa => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:49.760128 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0e30f-c294-4ac3-9118-63ebb1ad6c51 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:49.861659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0e30f-c294-4ac3-9118-63ebb1ad6c51 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:49.896960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2191b1-5115-4d2a-91c4-1134a8b36210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:49.897635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2191b1-5115-4d2a-91c4-1134a8b36210 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:49.897915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2191b1-5115-4d2a-91c4-1134a8b36210 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:49.898188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2191b1-5115-4d2a-91c4-1134a8b36210 None admin] PATH_INFO: `/v3/projects/64e3a120a396491882f930e29a8107fa/users/bbfe89da98284345b0237770cd2ac585/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:49.900135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2191b1-5115-4d2a-91c4-1134a8b36210 None admin] RBAC: Authorizing `identity:create_grant(project_id=64e3a120a396491882f930e29a8107fa, user_id=bbfe89da98284345b0237770cd2ac585, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:49.901576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2191b1-5115-4d2a-91c4-1134a8b36210 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:50.004949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 589/1192] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:37:49 2022] PUT /identity/v3/projects/64e3a120a396491882f930e29a8107fa/users/bbfe89da98284345b0237770cd2ac585/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:37:51.961833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589bf33-7484-4003-b179-81569ab213b0 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:51.962399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589bf33-7484-4003-b179-81569ab213b0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:51.962399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589bf33-7484-4003-b179-81569ab213b0 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:51.963701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 604/1193] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:37:51.968979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a2c19-306a-402f-a1f3-65daa1341bcc None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:51.969423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a2c19-306a-402f-a1f3-65daa1341bcc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:51.969515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a2c19-306a-402f-a1f3-65daa1341bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:51.987842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4a2a2c19-306a-402f-a1f3-65daa1341bcc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:37:51.989179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4a2a2c19-306a-402f-a1f3-65daa1341bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:37:52.000666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:52.062942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 590/1194] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:37:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:37:52.353275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589bf33-7484-4003-b179-81569ab213b0 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:52.357263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:52.475738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589bf33-7484-4003-b179-81569ab213b0 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:52.517315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26f8cc-71ab-4619-8ec4-879207348a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:52.517953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26f8cc-71ab-4619-8ec4-879207348a74 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:52.518229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26f8cc-71ab-4619-8ec4-879207348a74 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:52.518492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26f8cc-71ab-4619-8ec4-879207348a74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:52.530150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26f8cc-71ab-4619-8ec4-879207348a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-aa26f8cc-71ab-4619-8ec4-879207348a74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:37:52.538878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:37:52.540087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:37:52.541029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 605/1195] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:37:52.548665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a2c19-306a-402f-a1f3-65daa1341bcc None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:52.671614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a2c19-306a-402f-a1f3-65daa1341bcc None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:52.714102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:52.714699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:52.715006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:52.715280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:52.715947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:52.716250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:52.717634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:52.729843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 591/1196] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:37:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:52.735927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26f8cc-71ab-4619-8ec4-879207348a74 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:52.815844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:52.854855 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26f8cc-71ab-4619-8ec4-879207348a74 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:52.891904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403dbe3-8a6e-4389-859c-5beb8146cf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:52.892532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403dbe3-8a6e-4389-859c-5beb8146cf75 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:52.892800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403dbe3-8a6e-4389-859c-5beb8146cf75 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:52.893066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403dbe3-8a6e-4389-859c-5beb8146cf75 None admin] PATH_INFO: `/v3/users/bbfe89da98284345b0237770cd2ac585` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:52.894201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403dbe3-8a6e-4389-859c-5beb8146cf75 None admin] RBAC: Authorizing `identity:get_user(user_id=bbfe89da98284345b0237770cd2ac585)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:52.895759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403dbe3-8a6e-4389-859c-5beb8146cf75 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:52.897425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 606/1197] 10.222.0.94 () {56 vars in 1177 bytes} [Fri Aug 26 15:37:52 2022] GET /identity/v3/users/bbfe89da98284345b0237770cd2ac585 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:52.904575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:52.918892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:53.015422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9b6bf-5de4-471e-89e7-49b34ddadf08 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:53.051624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2928f-c9e3-4f86-93eb-8b95dd91420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:53.052193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2928f-c9e3-4f86-93eb-8b95dd91420d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:53.052461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2928f-c9e3-4f86-93eb-8b95dd91420d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:53.052743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2928f-c9e3-4f86-93eb-8b95dd91420d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:53.123185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2928f-c9e3-4f86-93eb-8b95dd91420d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:53.124625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2928f-c9e3-4f86-93eb-8b95dd91420d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:53.134592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:53.227138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 592/1198] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:37:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:37:53.233280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403dbe3-8a6e-4389-859c-5beb8146cf75 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:53.338594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403dbe3-8a6e-4389-859c-5beb8146cf75 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:53.369717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:53.374838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe210e-c5a1-4f82-8dab-a85bc0299cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:53.375554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe210e-c5a1-4f82-8dab-a85bc0299cd5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:53.375833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe210e-c5a1-4f82-8dab-a85bc0299cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:53.376103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe210e-c5a1-4f82-8dab-a85bc0299cd5 None admin] PATH_INFO: `/v3/projects/64e3a120a396491882f930e29a8107fa` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:53.377342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe210e-c5a1-4f82-8dab-a85bc0299cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=64e3a120a396491882f930e29a8107fa)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:53.378769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe210e-c5a1-4f82-8dab-a85bc0299cd5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:53.381378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 607/1199] 10.222.0.94 () {56 vars in 1186 bytes} [Fri Aug 26 15:37:53 2022] GET /identity/v3/projects/64e3a120a396491882f930e29a8107fa => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:37:53.387653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2928f-c9e3-4f86-93eb-8b95dd91420d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:37:53.497452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2928f-c9e3-4f86-93eb-8b95dd91420d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:37:53.507694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:37:53.534644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75682cc7-07ee-4307-95f4-453bb22f4b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:37:53.535275 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682cc7-07ee-4307-95f4-453bb22f4b04 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:37:53.535651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682cc7-07ee-4307-95f4-453bb22f4b04 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:37:53.536076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682cc7-07ee-4307-95f4-453bb22f4b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:37:53.536859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75682cc7-07ee-4307-95f4-453bb22f4b04 None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=64e3a120a396491882f930e29a8107fa, user.id=bbfe89da98284345b0237770cd2ac585) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:37:53.537156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75682cc7-07ee-4307-95f4-453bb22f4b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:37:53.538578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75682cc7-07ee-4307-95f4-453bb22f4b04 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:37:53.545042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 593/1200] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:37:53 2022] GET /identity/v3/role_assignments?user.id=bbfe89da98284345b0237770cd2ac585&scope.project.id=64e3a120a396491882f930e29a8107fa&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:38:03.545767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=14 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:17.152218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08641eab-26f3-4d83-b2b5-5da34745043d None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:17.152826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08641eab-26f3-4d83-b2b5-5da34745043d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:17.152826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08641eab-26f3-4d83-b2b5-5da34745043d None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:17.154321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 608/1201] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:17.160189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a840fa-fcf7-4b14-aa97-6b5f7bffd937 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:17.161221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a840fa-fcf7-4b14-aa97-6b5f7bffd937 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:17.161653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a840fa-fcf7-4b14-aa97-6b5f7bffd937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:17.180795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f4a840fa-fcf7-4b14-aa97-6b5f7bffd937 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:17.181996 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f4a840fa-fcf7-4b14-aa97-6b5f7bffd937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:17.263986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 594/1202] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:39:17.544104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08641eab-26f3-4d83-b2b5-5da34745043d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:17.663403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08641eab-26f3-4d83-b2b5-5da34745043d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:17.705369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:17.706003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:17.706271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:17.706536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:17.717453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:17.719020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:17.735355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:17.736751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:17.737913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:17.737913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:17.737913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:17.737913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 609/1203] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:39:17.743687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a840fa-fcf7-4b14-aa97-6b5f7bffd937 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:17.860106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a840fa-fcf7-4b14-aa97-6b5f7bffd937 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:17.895595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:17.902143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c4100-035c-4e25-b263-18c4b953541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:17.902790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4100-035c-4e25-b263-18c4b953541b None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:17.903108 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4100-035c-4e25-b263-18c4b953541b None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:17.903387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4100-035c-4e25-b263-18c4b953541b None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:17.904059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4100-035c-4e25-b263-18c4b953541b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:17.904371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4100-035c-4e25-b263-18c4b953541b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:17.905800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4100-035c-4e25-b263-18c4b953541b None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:17.922298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 595/1204] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:39:17.929270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:17.995270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:18.049970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2d6f1-cfa2-4908-83c9-29a51b753828 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:18.099560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc8aee-0e8a-48bb-93a8-732919fd3749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:18.100280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc8aee-0e8a-48bb-93a8-732919fd3749 None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:18.100617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc8aee-0e8a-48bb-93a8-732919fd3749 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:18.100892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc8aee-0e8a-48bb-93a8-732919fd3749 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:18.101656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc8aee-0e8a-48bb-93a8-732919fd3749 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:18.103352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc8aee-0e8a-48bb-93a8-732919fd3749 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:18.361600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 610/1205] 10.222.0.94 () {60 vars in 1132 bytes} [Fri Aug 26 15:39:17 2022] POST /identity/v3/users => generated 251 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:39:20.373915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780be441-d2f6-4512-9e9e-daa4faf0cced None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:20.374504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780be441-d2f6-4512-9e9e-daa4faf0cced None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:20.374504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780be441-d2f6-4512-9e9e-daa4faf0cced None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:20.375825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 596/1206] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:20.383865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919083b-9fed-4289-8f00-bbd8a13866eb None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:20.384634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919083b-9fed-4289-8f00-bbd8a13866eb None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:20.384634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919083b-9fed-4289-8f00-bbd8a13866eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:20.406178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6919083b-9fed-4289-8f00-bbd8a13866eb None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:20.407486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6919083b-9fed-4289-8f00-bbd8a13866eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:20.481681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 611/1207] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:39:20.774111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780be441-d2f6-4512-9e9e-daa4faf0cced None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:20.901949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780be441-d2f6-4512-9e9e-daa4faf0cced None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:20.939179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85ac48-9fe7-4193-b500-d5c9388d2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:20.939857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85ac48-9fe7-4193-b500-d5c9388d2360 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:20.940134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85ac48-9fe7-4193-b500-d5c9388d2360 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:20.940409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85ac48-9fe7-4193-b500-d5c9388d2360 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:20.953308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85ac48-9fe7-4193-b500-d5c9388d2360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-bc85ac48-9fe7-4193-b500-d5c9388d2360 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:39:20.959438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:20.960599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:20.961689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 597/1208] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:39:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:20.967392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919083b-9fed-4289-8f00-bbd8a13866eb None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:21.017164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:21.087012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919083b-9fed-4289-8f00-bbd8a13866eb None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:21.130682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:21.131471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:21.131810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:21.132092 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:21.132914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:21.133213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:21.135066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:21.145627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 612/1209] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:39:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:21.151758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85ac48-9fe7-4193-b500-d5c9388d2360 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:21.212572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:21.270587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85ac48-9fe7-4193-b500-d5c9388d2360 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:21.312410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:21.313178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:21.313473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:21.313747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:21.330249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:21.331717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:21.345891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:21.347228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:21.348511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:21.348511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:21.348511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:21.348511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 598/1210] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:39:21.353727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:21.463811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a1c6-1fd9-4335-b594-ee6d630f7710 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:21.505493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:21.527348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:21.527348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:21.527348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:21.527348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:21.527348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:21.527348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:21.527876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 613/1211] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:21.527970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:21.639459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dc7f1-1e99-454d-9034-60f633850582 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:21.675401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:21.676126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:21.677075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:21.677501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:21.685423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:21.687521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:21.689347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:39:21.702329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:21.703673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:21.704799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 599/1212] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:21.710014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:21.815957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e115240-0e8c-4108-89e1-e0d2ce02db23 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:21.852715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:21.853317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:21.853605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:21.853859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:21.854586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:21.854885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:21.856360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:21.876725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 614/1213] 10.222.0.94 () {56 vars in 1143 bytes} [Fri Aug 26 15:39:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:21.883603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:22.001175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:22.013296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09947a59-45e8-4917-82e9-37bda00cb044 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:22.047543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16d13d-d72e-4a00-8f65-abb620605ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:22.048145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16d13d-d72e-4a00-8f65-abb620605ed9 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:22.048413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16d13d-d72e-4a00-8f65-abb620605ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:22.048677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16d13d-d72e-4a00-8f65-abb620605ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:22.120732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16d13d-d72e-4a00-8f65-abb620605ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:22.122176 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16d13d-d72e-4a00-8f65-abb620605ed9 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:22.230935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 600/1214] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:39:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 15:39:22.237079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:22.310598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:22.351695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9d587-7c64-4df2-9181-8628a10aba23 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:22.388946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:22.389643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:22.389951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:22.390233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:22.401959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:22.403449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:22.418657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:22.425496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:22.426661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:22.426661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:22.426661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:22.426661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 615/1215] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:22.427001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16d13d-d72e-4a00-8f65-abb620605ed9 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:22.541117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16d13d-d72e-4a00-8f65-abb620605ed9 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:22.584631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:22.586215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:22.590497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:22.590497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:22.590497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:22.590497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:22.590497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:22.590497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:22.606037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 601/1216] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:22.616357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:22.745808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59598f99-d3eb-4561-ab94-fae92ae76584 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:22.782772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:22.783683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:22.783801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:22.783996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:22.791774 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:22.793255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:39:22.805081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:22.806283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:22.814285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 616/1217] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:39:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:22.815133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:22.841879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:22.928923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f18b3b-0fda-45da-8859-ec09987d65d8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:22.965615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99529d70-dc70-4cf9-96b7-66ddd75dbc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:22.966308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529d70-dc70-4cf9-96b7-66ddd75dbc5f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:22.966707 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529d70-dc70-4cf9-96b7-66ddd75dbc5f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:22.967020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99529d70-dc70-4cf9-96b7-66ddd75dbc5f None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:22.967942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99529d70-dc70-4cf9-96b7-66ddd75dbc5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:22.968336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99529d70-dc70-4cf9-96b7-66ddd75dbc5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:22.969972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99529d70-dc70-4cf9-96b7-66ddd75dbc5f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:22.982832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 602/1218] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:39:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:22.990120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:23.012870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:23.106370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53ba5b-50ce-4b76-b2a2-83631375ecf8 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:23.143622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3dbbe7-776d-4d7d-b92e-7a7431fd804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:23.144248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dbbe7-776d-4d7d-b92e-7a7431fd804e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:23.144521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dbbe7-776d-4d7d-b92e-7a7431fd804e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:23.144790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dbbe7-776d-4d7d-b92e-7a7431fd804e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:23.146419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dbbe7-776d-4d7d-b92e-7a7431fd804e None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=94983527f4f044b683921034aa1829b3) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:23.146725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dbbe7-776d-4d7d-b92e-7a7431fd804e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:23.148182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dbbe7-776d-4d7d-b92e-7a7431fd804e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:23.153957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 617/1219] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:39:22 2022] GET /identity/v3/role_assignments?user.id=94983527f4f044b683921034aa1829b3&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:39:23.156314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:23.298584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:25.154677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f151604-5ef5-46fb-81e7-f406946ebc46 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:25.155282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f151604-5ef5-46fb-81e7-f406946ebc46 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:25.155282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f151604-5ef5-46fb-81e7-f406946ebc46 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:25.156586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 603/1220] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:25.163125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949a5d-e597-4276-bec6-501b90394549 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:25.179565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949a5d-e597-4276-bec6-501b90394549 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:25.179565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949a5d-e597-4276-bec6-501b90394549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:25.184780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-34949a5d-e597-4276-bec6-501b90394549 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:25.186015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-34949a5d-e597-4276-bec6-501b90394549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:25.266486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 618/1221] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:39:25.539585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f151604-5ef5-46fb-81e7-f406946ebc46 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:25.649525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f151604-5ef5-46fb-81e7-f406946ebc46 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:25.686199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111d080-07d6-43e2-9885-0bb0309b6165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:25.686827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111d080-07d6-43e2-9885-0bb0309b6165 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:25.687132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111d080-07d6-43e2-9885-0bb0309b6165 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:25.687405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111d080-07d6-43e2-9885-0bb0309b6165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:25.698576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111d080-07d6-43e2-9885-0bb0309b6165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-4111d080-07d6-43e2-9885-0bb0309b6165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:39:25.703528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:25.704641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:25.705769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 604/1222] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:39:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:25.711187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34949a5d-e597-4276-bec6-501b90394549 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:25.816879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34949a5d-e597-4276-bec6-501b90394549 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:25.842835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:25.854503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:25.855210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:25.855492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:25.855760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:25.856434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] RBAC: Adding query filter params (name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:25.856809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:25.858187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:25.867827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 619/1223] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:39:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:39:25.873864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111d080-07d6-43e2-9885-0bb0309b6165 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:25.921045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:25.977896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111d080-07d6-43e2-9885-0bb0309b6165 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:26.012578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:26.013207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:26.013483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:26.013947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:26.024898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:26.026341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:26.040607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:26.057918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:26.059202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:26.059202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:26.059202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:26.059202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 605/1224] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:26.059494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:26.157533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5945b5d-ddc0-452a-850b-df2670137d11 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:26.193534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:26.194204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:26.194398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:26.194660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:26.195331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:26.195646 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:26.197006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:26.209897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 620/1225] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:26.215780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:26.315345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:26.322763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8e5e6-747e-4932-ab2e-6f3428d99f4d None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:26.357455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:26.358113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:26.358419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:26.358693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] PATH_INFO: `/v3/users/placement` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:26.368785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:26.370385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:39:26.383441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:26.384632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:26.385955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 606/1226] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:26.390586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:26.440303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:26.499009 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b29bb-1bc5-4dfb-9993-15d9cb5307b3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:26.537604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:26.538223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:26.538498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:26.538765 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:26.539536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:26.539859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:26.541287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:26.562769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 621/1227] 10.222.0.94 () {56 vars in 1143 bytes} [Fri Aug 26 15:39:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:26.569385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:26.686086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124d61c-1c2b-4dd0-9010-18dbcf74769a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:26.722905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:26.723555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:26.724160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:26.724277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:26.735416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:26.736888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:26.751474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:26.752665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:26.753559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:26.753559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:26.753559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:26.753559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 607/1228] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:26.759622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:26.870556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34330bb0-e328-4828-a9da-e7a4bd2e450d None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:26.878839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:26.911006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:26.911620 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:26.911890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:26.912190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:26.912858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:26.913133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:26.914493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:26.927195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 622/1229] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:26.933753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:26.945381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:27.050596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab7a2-d0a5-45ba-8e8d-632833d9552e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:27.087283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:27.087545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:27.087691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:27.087969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:27.095070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:27.096510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:39:27.108165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:27.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:27.110628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 608/1230] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:27.116950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:27.239520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a790f-6a43-47f3-a4f0-c8e6c04cb58e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:27.277218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004a9f9-5aab-412d-bcc9-e5eaa713052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:27.277898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004a9f9-5aab-412d-bcc9-e5eaa713052a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:27.278172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004a9f9-5aab-412d-bcc9-e5eaa713052a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:27.278465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004a9f9-5aab-412d-bcc9-e5eaa713052a None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:27.279233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004a9f9-5aab-412d-bcc9-e5eaa713052a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:27.279598 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004a9f9-5aab-412d-bcc9-e5eaa713052a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:27.281111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004a9f9-5aab-412d-bcc9-e5eaa713052a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:27.294940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 623/1231] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:39:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:27.301445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:27.375792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:27.415491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3927b-d712-4345-b057-4248f4526f79 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:27.456006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9766c-81bf-4c6b-9638-816fa882f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:27.456791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9766c-81bf-4c6b-9638-816fa882f8f5 None admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:27.457074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9766c-81bf-4c6b-9638-816fa882f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:27.457977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9766c-81bf-4c6b-9638-816fa882f8f5 None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/94983527f4f044b683921034aa1829b3/roles/b8b6b507d5284b84a25895b280b89cbd` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:27.476068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9766c-81bf-4c6b-9638-816fa882f8f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=94983527f4f044b683921034aa1829b3, role_id=b8b6b507d5284b84a25895b280b89cbd)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:27.477608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9766c-81bf-4c6b-9638-816fa882f8f5 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:27.687520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 609/1232] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:39:27 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/94983527f4f044b683921034aa1829b3/roles/b8b6b507d5284b84a25895b280b89cbd => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:39:29.691082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35ba15-765a-4801-bffd-00453cdf84f2 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:29.692085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35ba15-765a-4801-bffd-00453cdf84f2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:29.692085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35ba15-765a-4801-bffd-00453cdf84f2 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:29.693659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 624/1233] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:29.699242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f03f3-ea08-4181-bbc1-572db451b6cd None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:29.699914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f03f3-ea08-4181-bbc1-572db451b6cd None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:29.699914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f03f3-ea08-4181-bbc1-572db451b6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:29.720167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-082f03f3-ea08-4181-bbc1-572db451b6cd None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:29.721400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-082f03f3-ea08-4181-bbc1-572db451b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:29.798245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 610/1234] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:39:30.081878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35ba15-765a-4801-bffd-00453cdf84f2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:30.111447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:30.210341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35ba15-765a-4801-bffd-00453cdf84f2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:30.249195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01de6a-2f02-401e-a86d-51d1efafe932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:30.249841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01de6a-2f02-401e-a86d-51d1efafe932 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:30.250374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01de6a-2f02-401e-a86d-51d1efafe932 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:30.250593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01de6a-2f02-401e-a86d-51d1efafe932 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:30.263271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01de6a-2f02-401e-a86d-51d1efafe932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8e01de6a-2f02-401e-a86d-51d1efafe932 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:39:30.268040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:30.269257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:30.270727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 625/1235] 10.222.0.94 () {56 vars in 1102 bytes} [Fri Aug 26 15:39:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:30.276122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f03f3-ea08-4181-bbc1-572db451b6cd None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:30.335703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:30.394443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f03f3-ea08-4181-bbc1-572db451b6cd None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:30.432233 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:30.432853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:30.433149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:30.433676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:30.434406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] RBAC: Adding query filter params (name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:30.434915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:30.436861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:30.446590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 611/1236] 10.222.0.94 () {56 vars in 1103 bytes} [Fri Aug 26 15:39:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:30.452565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01de6a-2f02-401e-a86d-51d1efafe932 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:30.557346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:30.577899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01de6a-2f02-401e-a86d-51d1efafe932 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:30.617277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:30.617923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:30.618191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:30.618535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:30.630299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:30.631778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:30.647106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:30.650283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:30.651595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:30.651595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:30.651595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:30.651595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 626/1237] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:30.656653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:30.771173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110a462-2c0f-468a-81be-3196467ea935 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:30.783822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:30.812908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:30.813532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:30.813808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:30.814118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:30.814767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:30.815128 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:30.816717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:30.830757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 612/1238] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:30.838302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:30.951868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56ce0b-ed29-4c80-a233-2395912f640e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:30.998495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:30.999211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:30.999529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:30.999835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] PATH_INFO: `/v3/users/placement` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:31.007965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:31.009172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:31.010898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:39:31.022883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:31.024041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:31.025095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 627/1239] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:31.030125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:31.153513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44d3c9-e4d9-44e2-978a-3596531040df None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:31.190791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:31.193036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:31.193036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:31.193036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:31.193727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:31.194025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:31.195621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:31.221517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 613/1240] 10.222.0.94 () {56 vars in 1143 bytes} [Fri Aug 26 15:39:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:31.221720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:31.334246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859e158-d4cf-4181-ab06-6037400e182e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:31.372446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db646e7-7370-4b4b-8cb9-60321516e60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:31.373025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db646e7-7370-4b4b-8cb9-60321516e60f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:31.373307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db646e7-7370-4b4b-8cb9-60321516e60f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:31.373593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db646e7-7370-4b4b-8cb9-60321516e60f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:31.438179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:31.446409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db646e7-7370-4b4b-8cb9-60321516e60f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:31.447979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db646e7-7370-4b4b-8cb9-60321516e60f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:31.553728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 628/1241] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:39:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:39:31.559491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:31.589084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:31.664296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2c103-4742-43ee-a87b-e3f9c7a1fcb8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:31.702274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:31.702582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:31.703912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:31.703912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:31.715622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:31.717169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:31.732003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:31.733191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:31.734377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:31.734377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:31.734377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:31.734377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 614/1242] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:31.739824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db646e7-7370-4b4b-8cb9-60321516e60f None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:31.837634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:31.843593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db646e7-7370-4b4b-8cb9-60321516e60f None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:31.879738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:31.880608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:31.880884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:31.881150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:31.881796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:31.882090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:31.883488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:31.895371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 629/1243] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:31.901009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:31.983565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:32.005688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8708b52-4b22-444a-bfd6-7e12c95e8be0 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:32.045855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:32.046485 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:32.046763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:32.047066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:32.054660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:32.056311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:39:32.074188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:32.075539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:32.076444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 615/1244] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:32.081419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:32.188122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3400b59-935d-46b5-92d2-d05a3b1b4d22 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:32.223930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c0cd2-77fc-4615-a615-0f9c7861250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:32.224531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c0cd2-77fc-4615-a615-0f9c7861250f None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:32.224863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c0cd2-77fc-4615-a615-0f9c7861250f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:32.225131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c0cd2-77fc-4615-a615-0f9c7861250f None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:32.225837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c0cd2-77fc-4615-a615-0f9c7861250f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:32.226135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c0cd2-77fc-4615-a615-0f9c7861250f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:32.227593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c0cd2-77fc-4615-a615-0f9c7861250f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:32.240772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 630/1245] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:32.247034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:32.365232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e98a42-6ed0-4440-8dfd-05816c01121a None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:32.400363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6ea3d-392d-4125-9d81-91fa33810152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:32.400965 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6ea3d-392d-4125-9d81-91fa33810152 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:32.401239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6ea3d-392d-4125-9d81-91fa33810152 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:32.401506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6ea3d-392d-4125-9d81-91fa33810152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:32.402225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6ea3d-392d-4125-9d81-91fa33810152 None admin] RBAC: Adding query filter params (role.id=b8b6b507d5284b84a25895b280b89cbd, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=94983527f4f044b683921034aa1829b3) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:32.402525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6ea3d-392d-4125-9d81-91fa33810152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:32.403975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6ea3d-392d-4125-9d81-91fa33810152 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:32.410767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 616/1246] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:39:32 2022] GET /identity/v3/role_assignments?user.id=94983527f4f044b683921034aa1829b3&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=b8b6b507d5284b84a25895b280b89cbd => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:34.496526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666453-3e69-4b52-ac5b-8eea8b0fbd3b None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:34.497078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666453-3e69-4b52-ac5b-8eea8b0fbd3b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:34.497078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666453-3e69-4b52-ac5b-8eea8b0fbd3b None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:34.498442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 631/1247] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:34.505118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b24d-048a-4c83-b44a-fab42eff314b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:34.505658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b24d-048a-4c83-b44a-fab42eff314b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:34.505658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b24d-048a-4c83-b44a-fab42eff314b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:34.512128 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:34.526181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-da09b24d-048a-4c83-b44a-fab42eff314b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:34.527439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-da09b24d-048a-4c83-b44a-fab42eff314b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:34.620871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 617/1248] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:39:34.898526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77666453-3e69-4b52-ac5b-8eea8b0fbd3b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:34.919064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:35.009273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77666453-3e69-4b52-ac5b-8eea8b0fbd3b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:35.045521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2a10f-d923-46df-839d-b89f61e4f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:35.046244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2a10f-d923-46df-839d-b89f61e4f1e0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:35.046567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2a10f-d923-46df-839d-b89f61e4f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:35.046957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2a10f-d923-46df-839d-b89f61e4f1e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:35.058761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2a10f-d923-46df-839d-b89f61e4f1e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-aec2a10f-d923-46df-839d-b89f61e4f1e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:39:35.063644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:35.064849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:35.065768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 632/1249] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:39:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:35.071037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09b24d-048a-4c83-b44a-fab42eff314b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:35.177319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09b24d-048a-4c83-b44a-fab42eff314b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:35.209493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:35.215020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:35.215641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:35.215921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:35.216191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:35.216859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:35.217158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:35.218530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:35.229136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 618/1250] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:39:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:35.235203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2a10f-d923-46df-839d-b89f61e4f1e0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:35.321779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:35.345304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2a10f-d923-46df-839d-b89f61e4f1e0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:35.385116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:35.385775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:35.386069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:35.386342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:35.397825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:35.399380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:35.414618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:35.416551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:35.417873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:35.417873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:35.417873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:35.417873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 633/1251] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:35.421774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:35.531584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185063d4-0964-41d0-86f0-fc6735d150a8 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:35.567436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:35.568114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:35.568398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:35.568664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:35.569365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:35.569716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:35.571245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:35.584800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 619/1252] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:35.590836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:35.705280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad70ca2-8b9f-4a03-95f1-5c52d095120a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:35.735579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:35.744130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:35.744843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:35.745163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:35.745474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:35.753626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:35.755672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:39:35.767410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:35.771707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:35.773822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 634/1253] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:35.775181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:35.837483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:35.886018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb1a07-3dfb-4be8-a205-133d0d444b89 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:35.923815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:35.924473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:35.924751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:35.925021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:35.925983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:35.926455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:35.928486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:35.949994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 620/1254] 10.222.0.94 () {56 vars in 1143 bytes} [Fri Aug 26 15:39:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:35.956340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:36.084119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed341e-5082-4869-ac81-2f7f1f3e7b43 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:36.123325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3ae5f-bd46-4199-a7f6-2b65468479b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:36.123939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3ae5f-bd46-4199-a7f6-2b65468479b5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:36.124216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3ae5f-bd46-4199-a7f6-2b65468479b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:36.124482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3ae5f-bd46-4199-a7f6-2b65468479b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:36.184817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:36.197657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3ae5f-bd46-4199-a7f6-2b65468479b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:36.199140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3ae5f-bd46-4199-a7f6-2b65468479b5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:36.336062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 635/1255] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:39:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:39:36.342463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:36.456018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caf5bb-37e5-4f61-958a-9785cb49043e None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:36.470790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:36.500011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:36.500680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:36.500952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:36.501221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:36.516151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:36.517625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:36.533064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:36.534657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:36.536360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:36.536360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:36.536360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:36.536360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 621/1256] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:36.541321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3ae5f-bd46-4199-a7f6-2b65468479b5 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:36.652681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:36.674896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3ae5f-bd46-4199-a7f6-2b65468479b5 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:36.725964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:36.726485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:36.726763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:36.727067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:36.727817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:36.728120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:36.729481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:36.743725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 636/1257] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:36.749943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:36.890762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25add30b-ef0d-4b71-999a-a35bc5db3359 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:36.932584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:36.933225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:36.933501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:36.933772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:36.940683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:36.942440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:39:36.953973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:36.962298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:36.963469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 622/1258] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:39:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:36.963961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:37.083143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55e018-3ceb-4312-8d9c-211bb362906a None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:37.124098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15cc1-34a7-4147-9f5d-adc90d7edd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:37.124700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15cc1-34a7-4147-9f5d-adc90d7edd9e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:37.124977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15cc1-34a7-4147-9f5d-adc90d7edd9e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:37.125247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15cc1-34a7-4147-9f5d-adc90d7edd9e None admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:37.125941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15cc1-34a7-4147-9f5d-adc90d7edd9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:37.126240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15cc1-34a7-4147-9f5d-adc90d7edd9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:37.127693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15cc1-34a7-4147-9f5d-adc90d7edd9e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:37.141573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 637/1259] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:39:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:37.148944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:37.179261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:37.265608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138c289-907e-4cc2-aa33-f6b0b7b45a92 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:37.302345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6fe8c-f8d9-47cd-a7f5-6834c32c298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:37.302977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6fe8c-f8d9-47cd-a7f5-6834c32c298d None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:37.303257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6fe8c-f8d9-47cd-a7f5-6834c32c298d None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:37.303529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6fe8c-f8d9-47cd-a7f5-6834c32c298d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:37.304290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6fe8c-f8d9-47cd-a7f5-6834c32c298d None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=94983527f4f044b683921034aa1829b3) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:37.304638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6fe8c-f8d9-47cd-a7f5-6834c32c298d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:37.306076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6fe8c-f8d9-47cd-a7f5-6834c32c298d None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:37.311728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 623/1260] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:39:37 2022] GET /identity/v3/role_assignments?user.id=94983527f4f044b683921034aa1829b3&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:39.294702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97225792-c805-496c-85b1-0aeb51092917 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:39.295477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97225792-c805-496c-85b1-0aeb51092917 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:39.295477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97225792-c805-496c-85b1-0aeb51092917 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:39.296735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 638/1261] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:39.302843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2122e-f54a-4268-861f-665dc446d33b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:39.303402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2122e-f54a-4268-861f-665dc446d33b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:39.303500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2122e-f54a-4268-861f-665dc446d33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:39.323105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-a7e2122e-f54a-4268-861f-665dc446d33b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:39.324330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a7e2122e-f54a-4268-861f-665dc446d33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:39.395264 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 624/1262] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:39:39.675200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97225792-c805-496c-85b1-0aeb51092917 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:39.687109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:39.783301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97225792-c805-496c-85b1-0aeb51092917 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:39.820658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73341f-723e-449a-9b5d-803ba6603424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:39.821423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73341f-723e-449a-9b5d-803ba6603424 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:39.821701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73341f-723e-449a-9b5d-803ba6603424 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:39.821969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73341f-723e-449a-9b5d-803ba6603424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:39.833057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73341f-723e-449a-9b5d-803ba6603424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-2c73341f-723e-449a-9b5d-803ba6603424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:39:39.837692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:39.838825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:39.844766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 639/1263] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:39:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:39.845693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2122e-f54a-4268-861f-665dc446d33b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:39.865186 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:39.950248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2122e-f54a-4268-861f-665dc446d33b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:39.987080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:39.987772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:39.988894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:39.988894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:39.989148 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:39.989443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:39.990810 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:39.999739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 625/1264] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:39:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:40.006054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73341f-723e-449a-9b5d-803ba6603424 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:40.078624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:40.113986 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73341f-723e-449a-9b5d-803ba6603424 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:40.149442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:40.149679 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:40.149845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:40.150107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:40.160889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:40.162324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:40.177480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:40.178524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:40.179667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:40.179667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:40.179667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:40.179667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 640/1265] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:40.184685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:40.268124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:40.287642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5dd99-c00d-4995-b59a-1d59477074f1 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:40.321873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:40.322605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:40.322782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:40.323051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:40.323756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:40.324037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:40.325386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:40.337854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 626/1266] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:40.343873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:40.447837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523b4be-2436-4943-93d8-0e54f3aaad65 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:40.469296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:40.484296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:40.484934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:40.485326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:40.485456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:40.494235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:40.495784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:39:40.507635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:40.508828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:40.510246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 641/1267] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:40.514978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:40.623707 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811fb6f-461e-4d49-9e4a-697ac28c30b6 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:40.661256 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:40.661855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:40.662126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:40.662391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:40.663156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:40.663455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:40.664871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:40.674187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:40.683509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 627/1268] 10.222.0.94 () {56 vars in 1143 bytes} [Fri Aug 26 15:39:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:40.689977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:40.800538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03463641-91eb-4d36-9e96-b83eef1bdc79 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:40.837718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:40.838354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:40.838632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:40.838941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:40.851974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:40.853425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:40.868383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:40.869737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:40.870953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:40.870953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:40.870953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:40.870953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 642/1269] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:40.875542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:40.979605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0482-9f85-43ef-866d-b2e59951b5b7 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:41.016425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:41.017035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:41.017309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:41.017577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:41.018214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:41.018510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:41.021056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:41.032799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 628/1270] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:39:41.038696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:41.072655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:41.146567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264c83c-c9e2-4da8-a68c-26fabc84bff3 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:41.184356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:41.184996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:41.185271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:41.185537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:41.193531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:41.194991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:39:41.207508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:41.211087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:41.212194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 643/1271] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:39:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:41.214646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:41.275400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:41.322151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998693c1-3f66-4d9e-99f2-e32569969e19 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:41.357795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc8982-7fc6-494c-a4b3-0680433c8fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:41.358324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8982-7fc6-494c-a4b3-0680433c8fdb None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:41.358599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8982-7fc6-494c-a4b3-0680433c8fdb None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:41.359165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8982-7fc6-494c-a4b3-0680433c8fdb None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:41.359838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc8982-7fc6-494c-a4b3-0680433c8fdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:41.360136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc8982-7fc6-494c-a4b3-0680433c8fdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:41.361575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc8982-7fc6-494c-a4b3-0680433c8fdb None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:41.375049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 629/1272] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:39:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:41.382465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:41.481964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:41.491870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f4dfa-9df0-4680-9779-7829b997a56e None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:41.530416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c8888-f005-4109-bdf8-7ebef861c22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:41.531112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c8888-f005-4109-bdf8-7ebef861c22a None admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:41.531389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c8888-f005-4109-bdf8-7ebef861c22a None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:41.531658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c8888-f005-4109-bdf8-7ebef861c22a None admin] PATH_INFO: `/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/94983527f4f044b683921034aa1829b3/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:41.533529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c8888-f005-4109-bdf8-7ebef861c22a None admin] RBAC: Authorizing `identity:create_grant(project_id=5b63f7785db74906bb58fada43b9ff55, user_id=94983527f4f044b683921034aa1829b3, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:41.535082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c8888-f005-4109-bdf8-7ebef861c22a None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:41.692454 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 644/1273] 10.222.0.94 () {58 vars in 1439 bytes} [Fri Aug 26 15:39:41 2022] PUT /identity/v3/projects/5b63f7785db74906bb58fada43b9ff55/users/94983527f4f044b683921034aa1829b3/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:39:43.668299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a597-9987-4065-b8e5-eaa7dc44ddb7 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:43.668829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a597-9987-4065-b8e5-eaa7dc44ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:43.668942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a597-9987-4065-b8e5-eaa7dc44ddb7 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:43.670240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 630/1274] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:43.676626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017eb27-8329-44ba-b55d-8ab073171e3a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:43.677086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017eb27-8329-44ba-b55d-8ab073171e3a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:43.677228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017eb27-8329-44ba-b55d-8ab073171e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:43.696511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2017eb27-8329-44ba-b55d-8ab073171e3a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:43.697737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2017eb27-8329-44ba-b55d-8ab073171e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:43.773967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 645/1275] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:39:44.048365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109a597-9987-4065-b8e5-eaa7dc44ddb7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:44.173560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109a597-9987-4065-b8e5-eaa7dc44ddb7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:44.193634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:44.212661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a8120-7345-4966-9c6f-478c9b2cb8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:44.213292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a8120-7345-4966-9c6f-478c9b2cb8b2 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:44.213568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a8120-7345-4966-9c6f-478c9b2cb8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:44.213838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a8120-7345-4966-9c6f-478c9b2cb8b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:44.224095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a8120-7345-4966-9c6f-478c9b2cb8b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ac1a8120-7345-4966-9c6f-478c9b2cb8b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 15:39:44.228914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:44.230053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:44.231109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 631/1276] 10.222.0.94 () {56 vars in 1096 bytes} [Fri Aug 26 15:39:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 15:39:44.236453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017eb27-8329-44ba-b55d-8ab073171e3a None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:44.325001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:44.351908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017eb27-8329-44ba-b55d-8ab073171e3a None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:44.391078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:44.391691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:44.391962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:44.392225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:44.392909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:44.393197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:44.394611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:44.402995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 646/1277] 10.222.0.94 () {56 vars in 1099 bytes} [Fri Aug 26 15:39:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:44.409097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a8120-7345-4966-9c6f-478c9b2cb8b2 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:44.515222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a8120-7345-4966-9c6f-478c9b2cb8b2 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:44.548961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:44.549597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:44.549872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:44.554352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:44.569191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:44.570705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:44.586562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:44.594750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:44.595903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:44.595903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:44.595903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:44.595903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 632/1278] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:44.596198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:44.698469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4e363-e0ef-4936-aca6-c03bf0a76996 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:44.733122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:44.733981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:44.734253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:44.734517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:44.735258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:44.735552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:44.736930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:44.748637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 647/1279] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:44.754896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:44.787799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:44.874379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785c40d-c856-4363-9fc2-2a40de3e24c2 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:44.913564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:44.914199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:44.914478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:44.914749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:44.922651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:44.924189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 15:39:44.935472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:44.936605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 15:39:44.937794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:44.939078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 633/1280] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:44.944143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:44.949995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:45.057728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8464d29-63dd-4cab-bfcd-14c3925ff1c0 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:45.102334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:45.102974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:45.103255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:45.103521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:45.104244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:45.104545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:45.105966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:45.129545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 648/1281] 10.222.0.94 () {56 vars in 1143 bytes} [Fri Aug 26 15:39:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:45.132534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:45.242999 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:45.252895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabcf9c-97fe-4edd-b481-fba5cc720524 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:45.294638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe2c82-454a-428f-a737-ccf4a2edc2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:45.295692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe2c82-454a-428f-a737-ccf4a2edc2cd None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:45.296183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe2c82-454a-428f-a737-ccf4a2edc2cd None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:45.296658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe2c82-454a-428f-a737-ccf4a2edc2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:45.395014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe2c82-454a-428f-a737-ccf4a2edc2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:45.396530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe2c82-454a-428f-a737-ccf4a2edc2cd None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:45.514595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:45.525861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 634/1282] 10.222.0.94 () {58 vars in 1282 bytes} [Fri Aug 26 15:39:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 15:39:45.532485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:45.625217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:45.666265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14560e1-2500-457c-b310-e1e1d893eb11 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:45.703824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:45.704484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:45.704721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:45.704987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:45.716257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:45.718355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 15:39:45.733772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:45.734785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 15:39:45.736240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 15:39:45.736240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 15:39:45.736240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:45.736240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 649/1283] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:45.741174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe2c82-454a-428f-a737-ccf4a2edc2cd None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:45.845695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe2c82-454a-428f-a737-ccf4a2edc2cd None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:45.882628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:45.883273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:45.883550 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:45.883819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:45.884466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:45.884764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:45.886340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:45.898096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 635/1284] 10.222.0.94 () {56 vars in 1109 bytes} [Fri Aug 26 15:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:45.903927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:46.008470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075915-34ce-47b6-942e-e48a3a9750ac None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:46.025685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:46.047519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:46.048150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:46.048425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:46.048691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:46.056214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:46.057649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 15:39:46.070127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:46.071255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: ERROR keystone.server.flask.application Aug 26 15:39:46.072302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 650/1285] 10.222.0.94 () {56 vars in 1111 bytes} [Fri Aug 26 15:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:46.077350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:46.133341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:46.185992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab2663-2b65-4d7f-b2db-1bb5cfd3241f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:46.222250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a92a32-919f-4d30-bd2e-005dd3ab90c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:46.223018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a92a32-919f-4d30-bd2e-005dd3ab90c8 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:46.223175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a92a32-919f-4d30-bd2e-005dd3ab90c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:46.223443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a92a32-919f-4d30-bd2e-005dd3ab90c8 None admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:46.224158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a92a32-919f-4d30-bd2e-005dd3ab90c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:46.224456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a92a32-919f-4d30-bd2e-005dd3ab90c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:46.225877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a92a32-919f-4d30-bd2e-005dd3ab90c8 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:46.238683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 636/1286] 10.222.0.94 () {56 vars in 1148 bytes} [Fri Aug 26 15:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:46.244984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:46.352579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e182089-035f-4af1-b6b1-953bd50437d5 None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:46.388556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d852a-7844-413c-a7f6-90a7f9a3ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:46.389163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d852a-7844-413c-a7f6-90a7f9a3ce2b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:46.389437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d852a-7844-413c-a7f6-90a7f9a3ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:46.389706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d852a-7844-413c-a7f6-90a7f9a3ce2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:46.390431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d852a-7844-413c-a7f6-90a7f9a3ce2b None admin] RBAC: Adding query filter params (role.id=46d63829902d4da196c41ff99b2ac0c5, scope.project.id=5b63f7785db74906bb58fada43b9ff55, user.id=94983527f4f044b683921034aa1829b3) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:46.390732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d852a-7844-413c-a7f6-90a7f9a3ce2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:46.392205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d852a-7844-413c-a7f6-90a7f9a3ce2b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:46.399306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 651/1287] 10.222.0.94 () {56 vars in 1374 bytes} [Fri Aug 26 15:39:46 2022] GET /identity/v3/role_assignments?user.id=94983527f4f044b683921034aa1829b3&scope.project.id=5b63f7785db74906bb58fada43b9ff55&role.id=46d63829902d4da196c41ff99b2ac0c5 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:48.431522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6458d4-b0ad-4964-8b65-dc35c8dc4766 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:48.432105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6458d4-b0ad-4964-8b65-dc35c8dc4766 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:48.432105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6458d4-b0ad-4964-8b65-dc35c8dc4766 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:48.433399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 637/1288] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:48.439180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292cc52-bdd0-40e0-ae86-7ab4474f6849 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:48.439928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292cc52-bdd0-40e0-ae86-7ab4474f6849 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:48.440129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292cc52-bdd0-40e0-ae86-7ab4474f6849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:48.459813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-0292cc52-bdd0-40e0-ae86-7ab4474f6849 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:48.461059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0292cc52-bdd0-40e0-ae86-7ab4474f6849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:48.494354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:48.533912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 652/1289] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:39:48.809649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6458d4-b0ad-4964-8b65-dc35c8dc4766 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:48.918785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6458d4-b0ad-4964-8b65-dc35c8dc4766 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:48.937194 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:48.955620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:48.956247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:48.956514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:48.956776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] PATH_INFO: `/v3/services/placement` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:48.957347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:48.958733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:48.967491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:48.968738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:48.970121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 638/1290] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:39:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:48.974948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292cc52-bdd0-40e0-ae86-7ab4474f6849 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:49.086946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292cc52-bdd0-40e0-ae86-7ab4474f6849 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:49.122137 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04664afc-1d53-4eb2-998a-eedece5fe193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:49.122748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664afc-1d53-4eb2-998a-eedece5fe193 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:49.123060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664afc-1d53-4eb2-998a-eedece5fe193 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:49.123333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664afc-1d53-4eb2-998a-eedece5fe193 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:49.124034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04664afc-1d53-4eb2-998a-eedece5fe193 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:49.124293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04664afc-1d53-4eb2-998a-eedece5fe193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:49.125626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04664afc-1d53-4eb2-998a-eedece5fe193 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:49.131522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 653/1291] 10.222.0.94 () {56 vars in 1116 bytes} [Fri Aug 26 15:39:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:49.137617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:49.247544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bf5cd-58bb-4e66-b5a4-cc07abbb5d1f None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:49.282851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a8589-bf53-4e7a-b222-28632871d803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:49.284458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8589-bf53-4e7a-b222-28632871d803 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:49.284458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8589-bf53-4e7a-b222-28632871d803 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:49.284458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8589-bf53-4e7a-b222-28632871d803 None admin] PATH_INFO: `/v3/services` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:49.284943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a8589-bf53-4e7a-b222-28632871d803 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:49.285374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a8589-bf53-4e7a-b222-28632871d803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:49.286741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a8589-bf53-4e7a-b222-28632871d803 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:49.291703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 639/1292] 10.222.0.94 () {56 vars in 1116 bytes} [Fri Aug 26 15:39:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:51.266707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738669a-5a9e-4512-bb42-56534d7f01f6 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:51.267377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738669a-5a9e-4512-bb42-56534d7f01f6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:51.267377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738669a-5a9e-4512-bb42-56534d7f01f6 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:51.269265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 654/1293] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:51.275217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e331aa-cf24-417a-b59e-39fa1f4f7464 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:51.282095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e331aa-cf24-417a-b59e-39fa1f4f7464 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:51.282095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e331aa-cf24-417a-b59e-39fa1f4f7464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:51.298296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-b1e331aa-cf24-417a-b59e-39fa1f4f7464 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:51.299535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b1e331aa-cf24-417a-b59e-39fa1f4f7464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:51.330470 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:51.371001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 640/1294] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:39:51.656051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738669a-5a9e-4512-bb42-56534d7f01f6 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:51.689999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:51.764539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738669a-5a9e-4512-bb42-56534d7f01f6 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:51.799823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d49fac-cd6f-4d50-9c75-891fceb3054f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:51.801215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49fac-cd6f-4d50-9c75-891fceb3054f None admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:51.801494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49fac-cd6f-4d50-9c75-891fceb3054f None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:51.801764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49fac-cd6f-4d50-9c75-891fceb3054f None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:51.802407 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d49fac-cd6f-4d50-9c75-891fceb3054f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:51.803841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d49fac-cd6f-4d50-9c75-891fceb3054f None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:51.920630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 655/1295] 10.222.0.94 () {60 vars in 1141 bytes} [Fri Aug 26 15:39:51 2022] POST /identity/v3/services => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:39:54.104961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbd364-e062-4a4b-9c95-c4529a486a03 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:54.105535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbd364-e062-4a4b-9c95-c4529a486a03 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:54.105535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbd364-e062-4a4b-9c95-c4529a486a03 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:54.106858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 641/1296] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:54.113433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599523f-e526-44be-b3c1-e3d79240ba7d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:54.114333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599523f-e526-44be-b3c1-e3d79240ba7d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:54.114333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599523f-e526-44be-b3c1-e3d79240ba7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:54.136518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4599523f-e526-44be-b3c1-e3d79240ba7d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:54.137915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4599523f-e526-44be-b3c1-e3d79240ba7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:54.218820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 656/1297] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 15:39:54.496643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbd364-e062-4a4b-9c95-c4529a486a03 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:54.641102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbd364-e062-4a4b-9c95-c4529a486a03 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:54.678103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:54.678728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:54.679031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:54.679292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:54.679872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:54.681634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:54.693493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:54.694988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:54.695973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 642/1298] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:39:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:54.701674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599523f-e526-44be-b3c1-e3d79240ba7d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:54.725110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:54.825625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599523f-e526-44be-b3c1-e3d79240ba7d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:54.873485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:54.874100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:54.874380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:54.874652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:54.875366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] RBAC: Adding query filter params (name=placement) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:54.875673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:54.877390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:54.884224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 657/1299] 10.222.0.94 () {56 vars in 1116 bytes} [Fri Aug 26 15:39:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:54.894288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:54.933539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:55.018434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d47b1-bb09-4f95-bb6d-e934b00a14c3 None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:55.054979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8161b6-7ede-4a6e-ba82-b7b86e24860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:55.055585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8161b6-7ede-4a6e-ba82-b7b86e24860a None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:55.055863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8161b6-7ede-4a6e-ba82-b7b86e24860a None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:55.056134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8161b6-7ede-4a6e-ba82-b7b86e24860a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:55.056837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8161b6-7ede-4a6e-ba82-b7b86e24860a None admin] RBAC: Adding query filter params (interface=public, service_id=a535c64f7317415d9d16dc0ca86179ae, region_id=RegionOne) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:55.057134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8161b6-7ede-4a6e-ba82-b7b86e24860a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:55.058502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8161b6-7ede-4a6e-ba82-b7b86e24860a None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:55.064899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 643/1300] 10.222.0.94 () {56 vars in 1251 bytes} [Fri Aug 26 15:39:54 2022] GET /identity/v3/endpoints?service_id=a535c64f7317415d9d16dc0ca86179ae&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:55.071684 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:55.179101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af280f74-46c5-42fd-9ada-d7d678b43a2b None admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:55.182953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:55.217894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1bf0a-f43d-4d10-8b2d-86b8701dbc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:55.218509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1bf0a-f43d-4d10-8b2d-86b8701dbc9c None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:55.218786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1bf0a-f43d-4d10-8b2d-86b8701dbc9c None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:55.219086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1bf0a-f43d-4d10-8b2d-86b8701dbc9c None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:55.219819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1bf0a-f43d-4d10-8b2d-86b8701dbc9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:55.221102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1bf0a-f43d-4d10-8b2d-86b8701dbc9c None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:55.228517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 658/1301] 10.222.0.94 () {56 vars in 1087 bytes} [Fri Aug 26 15:39:55 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:39:57.187262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acb6b7-0bcc-44e5-b37a-594f9fa0c0d5 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:57.187797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acb6b7-0bcc-44e5-b37a-594f9fa0c0d5 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:57.187797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acb6b7-0bcc-44e5-b37a-594f9fa0c0d5 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:57.189157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 644/1302] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:39:57.195567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15de22-f353-4643-ae9e-e29523d3b8a4 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:57.196046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15de22-f353-4643-ae9e-e29523d3b8a4 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:57.196141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15de22-f353-4643-ae9e-e29523d3b8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:57.214140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3e15de22-f353-4643-ae9e-e29523d3b8a4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:39:57.222438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3e15de22-f353-4643-ae9e-e29523d3b8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:39:57.290011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 659/1303] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:39:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 15:39:57.570837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acb6b7-0bcc-44e5-b37a-594f9fa0c0d5 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:57.671305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:57.679515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acb6b7-0bcc-44e5-b37a-594f9fa0c0d5 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:57.716611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:57.717253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:57.717253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:57.717787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] PATH_INFO: `/v3/services/placement` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:57.718379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:57.719823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 15:39:57.728865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 15:39:57.730012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application with Lock( Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self._enter() Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application return self.creator() Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application created_value = creator( Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: ERROR keystone.server.flask.application Aug 26 15:39:57.730879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 645/1304] 10.222.0.94 () {56 vars in 1117 bytes} [Fri Aug 26 15:39:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 15:39:57.737878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15de22-f353-4643-ae9e-e29523d3b8a4 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:57.824735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:39:57.847312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15de22-f353-4643-ae9e-e29523d3b8a4 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:57.884925 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c36462-3bd8-408b-8c97-6c091baa6230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:57.885563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36462-3bd8-408b-8c97-6c091baa6230 None admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:57.885842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36462-3bd8-408b-8c97-6c091baa6230 None admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:57.886112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c36462-3bd8-408b-8c97-6c091baa6230 None admin] PATH_INFO: `/v3/services` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:57.886747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c36462-3bd8-408b-8c97-6c091baa6230 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:39:57.887080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c36462-3bd8-408b-8c97-6c091baa6230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:57.888513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c36462-3bd8-408b-8c97-6c091baa6230 None admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:57.894224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 660/1305] 10.222.0.94 () {56 vars in 1116 bytes} [Fri Aug 26 15:39:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:39:57.901593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:39:58.010519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b0cd4-4893-4cc0-85d8-b768823083ac None admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:39:58.048101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31599304-5755-40c4-9c19-237b36c16d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:39:58.048722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31599304-5755-40c4-9c19-237b36c16d41 None admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:39:58.049000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31599304-5755-40c4-9c19-237b36c16d41 None admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:39:58.049335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31599304-5755-40c4-9c19-237b36c16d41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:39:58.050680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31599304-5755-40c4-9c19-237b36c16d41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:39:58.052125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31599304-5755-40c4-9c19-237b36c16d41 None admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:39:58.161522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 646/1306] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:39:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:40:07.890786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=30 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:40:08.155745 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:11.020654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75da1c-1f41-491e-818f-e9bf1ffe1902 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:11.021606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75da1c-1f41-491e-818f-e9bf1ffe1902 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:11.022034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75da1c-1f41-491e-818f-e9bf1ffe1902 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:11.030569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8437b3d-fc41-4068-8b58-d124fc94e0b9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:11.031514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8437b3d-fc41-4068-8b58-d124fc94e0b9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:11.031945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8437b3d-fc41-4068-8b58-d124fc94e0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:11.036997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 661/1307] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:11.128463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c8437b3d-fc41-4068-8b58-d124fc94e0b9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:11.129945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c8437b3d-fc41-4068-8b58-d124fc94e0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:11.311474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 647/1308] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:11.711058 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da93c-9a4c-43b5-ab6c-d5c5aaae9dab None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:11.712187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da93c-9a4c-43b5-ab6c-d5c5aaae9dab None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:11.712697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da93c-9a4c-43b5-ab6c-d5c5aaae9dab None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:11.714521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 662/1309] 10.222.0.94 () {54 vars in 965 bytes} [Fri Aug 26 15:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:11.721547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7d8b-a764-46ca-ac6f-d851678ab329 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:11.722250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7d8b-a764-46ca-ac6f-d851678ab329 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:11.722736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7d8b-a764-46ca-ac6f-d851678ab329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:11.812880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-2b8e7d8b-a764-46ca-ac6f-d851678ab329 None None] MFA Rules not processed for user `b7ad758153334ec28940156d09495ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:11.814461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2b8e7d8b-a764-46ca-ac6f-d851678ab329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:11.842971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:11.935213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184da93c-9a4c-43b5-ab6c-d5c5aaae9dab None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:11.937357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-184da93c-9a4c-43b5-ab6c-d5c5aaae9dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:11.940746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 648/1310] 10.222.0.94 () {58 vars in 1062 bytes} [Fri Aug 26 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:48:12.099138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184da93c-9a4c-43b5-ab6c-d5c5aaae9dab None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:12.220695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc622b3-e8aa-4ab6-b75b-7e0684db1f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:12.221334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc622b3-e8aa-4ab6-b75b-7e0684db1f87 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:12.221611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc622b3-e8aa-4ab6-b75b-7e0684db1f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:12.222017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc622b3-e8aa-4ab6-b75b-7e0684db1f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:12.223815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4fc622b3-e8aa-4ab6-b75b-7e0684db1f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:12.293098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:12.357253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc622b3-e8aa-4ab6-b75b-7e0684db1f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:12.359219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc622b3-e8aa-4ab6-b75b-7e0684db1f87 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:12.382371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 663/1311] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:14.862625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495773-3d60-4156-ae05-ab4a5b435ea7 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:14.863260 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495773-3d60-4156-ae05-ab4a5b435ea7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:14.863260 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495773-3d60-4156-ae05-ab4a5b435ea7 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:14.867165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 649/1312] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:14.871430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7207bb-04de-4ca8-852f-6d3d55ac8368 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:14.871917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7207bb-04de-4ca8-852f-6d3d55ac8368 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:14.872028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7207bb-04de-4ca8-852f-6d3d55ac8368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:14.962056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2d7207bb-04de-4ca8-852f-6d3d55ac8368 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:14.963485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2d7207bb-04de-4ca8-852f-6d3d55ac8368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:15.069170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 664/1313] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:48:15.354166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5495773-3d60-4156-ae05-ab4a5b435ea7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:15.355763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b5495773-3d60-4156-ae05-ab4a5b435ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:15.460898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5495773-3d60-4156-ae05-ab4a5b435ea7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:15.533024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:15.535678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da87dd-0fa0-406a-8e65-c0c218aa5c68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:15.536286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da87dd-0fa0-406a-8e65-c0c218aa5c68 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:15.536567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da87dd-0fa0-406a-8e65-c0c218aa5c68 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:15.536838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da87dd-0fa0-406a-8e65-c0c218aa5c68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:15.538510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b2da87dd-0fa0-406a-8e65-c0c218aa5c68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:15.648567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da87dd-0fa0-406a-8e65-c0c218aa5c68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:15.650770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da87dd-0fa0-406a-8e65-c0c218aa5c68 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:15.669084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 650/1314] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:18.148039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6207ea-dfa3-456f-8431-10f37a9cf8af None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:18.148584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6207ea-dfa3-456f-8431-10f37a9cf8af None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:18.148699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6207ea-dfa3-456f-8431-10f37a9cf8af None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:18.151102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 665/1315] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:18.156700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5bf93-4176-4f13-bad7-d9e518b83c05 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:18.157166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5bf93-4176-4f13-bad7-d9e518b83c05 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:18.157266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5bf93-4176-4f13-bad7-d9e518b83c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:18.228537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e5f5bf93-4176-4f13-bad7-d9e518b83c05 None None] MFA Rules not processed for user `c7bb81025bd540828ef536c262646131`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:18.229762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e5f5bf93-4176-4f13-bad7-d9e518b83c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:18.283043 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:18.329771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 651/1316] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:18.618563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6207ea-dfa3-456f-8431-10f37a9cf8af None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:18.620211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ed6207ea-dfa3-456f-8431-10f37a9cf8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:18.630072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:18.727969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6207ea-dfa3-456f-8431-10f37a9cf8af None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:18.805322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35115c12-d2cc-40e9-b703-47202952dd0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:18.805940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35115c12-d2cc-40e9-b703-47202952dd0f service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:18.806226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35115c12-d2cc-40e9-b703-47202952dd0f service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:18.806499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35115c12-d2cc-40e9-b703-47202952dd0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:18.808211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-35115c12-d2cc-40e9-b703-47202952dd0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:18.873229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:18.922107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35115c12-d2cc-40e9-b703-47202952dd0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:18.923565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35115c12-d2cc-40e9-b703-47202952dd0f service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:18.941253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 666/1317] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:21.760562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff000-b838-4bc7-a61f-b4eca44eab19 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:21.761164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff000-b838-4bc7-a61f-b4eca44eab19 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:21.761164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff000-b838-4bc7-a61f-b4eca44eab19 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:21.762484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 652/1318] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:21.767887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765540cf-9b9e-4329-9e8b-9e5e303214d3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:21.768301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765540cf-9b9e-4329-9e8b-9e5e303214d3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:21.768431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765540cf-9b9e-4329-9e8b-9e5e303214d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:21.835578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-765540cf-9b9e-4329-9e8b-9e5e303214d3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:21.836798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-765540cf-9b9e-4329-9e8b-9e5e303214d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:21.932518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 667/1319] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:22.222147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff000-b838-4bc7-a61f-b4eca44eab19 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:22.223733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-41aff000-b838-4bc7-a61f-b4eca44eab19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:22.326576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff000-b838-4bc7-a61f-b4eca44eab19 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:22.398747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f13d7-c88c-4e18-9ca8-5b383649fd42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:22.399962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f13d7-c88c-4e18-9ca8-5b383649fd42 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:22.400231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f13d7-c88c-4e18-9ca8-5b383649fd42 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:22.400525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f13d7-c88c-4e18-9ca8-5b383649fd42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:22.402121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2d2f13d7-c88c-4e18-9ca8-5b383649fd42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:22.409573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:22.510152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f13d7-c88c-4e18-9ca8-5b383649fd42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:22.511529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f13d7-c88c-4e18-9ca8-5b383649fd42 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:22.529688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 653/1320] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:25.589332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fcf82-02d8-4884-abc8-18342ed37b65 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:25.589911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fcf82-02d8-4884-abc8-18342ed37b65 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:25.589911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fcf82-02d8-4884-abc8-18342ed37b65 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:25.591303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 668/1321] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:25.597057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a49cb-b775-43bf-aff8-11953c0e0eaa None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:25.597516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a49cb-b775-43bf-aff8-11953c0e0eaa None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:25.597617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a49cb-b775-43bf-aff8-11953c0e0eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:25.666495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-985a49cb-b775-43bf-aff8-11953c0e0eaa None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:25.667717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-985a49cb-b775-43bf-aff8-11953c0e0eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:25.728319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:25.769881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 654/1322] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:26.071498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fcf82-02d8-4884-abc8-18342ed37b65 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:26.073263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ca9fcf82-02d8-4884-abc8-18342ed37b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:26.095301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:26.188216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fcf82-02d8-4884-abc8-18342ed37b65 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:26.272671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b263cc-07fc-4a82-84fa-e2672ff00a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:26.273285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b263cc-07fc-4a82-84fa-e2672ff00a4f service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:26.273561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b263cc-07fc-4a82-84fa-e2672ff00a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:26.274515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b263cc-07fc-4a82-84fa-e2672ff00a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:26.276109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-29b263cc-07fc-4a82-84fa-e2672ff00a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:26.356156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:26.393525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b263cc-07fc-4a82-84fa-e2672ff00a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:26.395070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b263cc-07fc-4a82-84fa-e2672ff00a4f service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:26.413856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 669/1323] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:28.463617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa57012-b061-4a81-9ae1-63cfd7425335 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:28.464142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa57012-b061-4a81-9ae1-63cfd7425335 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:28.464351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa57012-b061-4a81-9ae1-63cfd7425335 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:28.465681 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 655/1324] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:28.471915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60e65e-26c8-4059-8843-3b7872924c63 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:28.472381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60e65e-26c8-4059-8843-3b7872924c63 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:28.472594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60e65e-26c8-4059-8843-3b7872924c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:28.542741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ca60e65e-26c8-4059-8843-3b7872924c63 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:28.544029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ca60e65e-26c8-4059-8843-3b7872924c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:28.644536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 670/1325] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:28.950558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9c72e-1223-4d4f-a7d2-640528bfcbf6 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:28.950965 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9c72e-1223-4d4f-a7d2-640528bfcbf6 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:28.951245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9c72e-1223-4d4f-a7d2-640528bfcbf6 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:28.952592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 656/1326] 10.222.0.94 () {54 vars in 965 bytes} [Fri Aug 26 15:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:28.958700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb6352-307b-4bec-ad7f-fd101e5a6aaf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:28.959088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb6352-307b-4bec-ad7f-fd101e5a6aaf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:28.959477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb6352-307b-4bec-ad7f-fd101e5a6aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:28.999501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:29.047423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-22cb6352-307b-4bec-ad7f-fd101e5a6aaf None None] MFA Rules not processed for user `b7ad758153334ec28940156d09495ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:29.048633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-22cb6352-307b-4bec-ad7f-fd101e5a6aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:29.134376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 671/1327] 10.222.0.94 () {58 vars in 1062 bytes} [Fri Aug 26 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:48:29.143389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9c72e-1223-4d4f-a7d2-640528bfcbf6 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:29.144943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-83d9c72e-1223-4d4f-a7d2-640528bfcbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:29.259429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9c72e-1223-4d4f-a7d2-640528bfcbf6 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:29.338565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566e202-12c5-4cec-8e32-24f6c1c8ee3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:29.339384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566e202-12c5-4cec-8e32-24f6c1c8ee3a service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:29.339632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566e202-12c5-4cec-8e32-24f6c1c8ee3a service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:29.339903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566e202-12c5-4cec-8e32-24f6c1c8ee3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:29.341526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7566e202-12c5-4cec-8e32-24f6c1c8ee3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:29.360409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:29.461138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566e202-12c5-4cec-8e32-24f6c1c8ee3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:29.463677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566e202-12c5-4cec-8e32-24f6c1c8ee3a service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:29.483787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 657/1328] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:31.801154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce48b8-5a89-44a7-bee5-764387181314 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:31.801673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce48b8-5a89-44a7-bee5-764387181314 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:31.801673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce48b8-5a89-44a7-bee5-764387181314 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:31.802983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 672/1329] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:31.809556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f427a-f515-4c8c-b2c4-f1379cd248b2 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:31.810052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f427a-f515-4c8c-b2c4-f1379cd248b2 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:31.810196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f427a-f515-4c8c-b2c4-f1379cd248b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:31.884615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-bf8f427a-f515-4c8c-b2c4-f1379cd248b2 None None] MFA Rules not processed for user `c7bb81025bd540828ef536c262646131`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:31.885848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bf8f427a-f515-4c8c-b2c4-f1379cd248b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:31.959577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:31.995028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 658/1330] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:32.283877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce48b8-5a89-44a7-bee5-764387181314 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:32.285420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-90ce48b8-5a89-44a7-bee5-764387181314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:32.390705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce48b8-5a89-44a7-bee5-764387181314 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:32.411970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:32.467532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7eda9-0288-4f4a-9751-d851226bb8ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:32.468134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7eda9-0288-4f4a-9751-d851226bb8ac service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:32.468405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7eda9-0288-4f4a-9751-d851226bb8ac service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:32.468667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7eda9-0288-4f4a-9751-d851226bb8ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:32.470509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c0a7eda9-0288-4f4a-9751-d851226bb8ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:32.583914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7eda9-0288-4f4a-9751-d851226bb8ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:32.585391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7eda9-0288-4f4a-9751-d851226bb8ac service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:32.606046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 673/1331] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:35.191677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30635c10-4737-4db0-8fa2-19b2e0032959 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:35.192219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30635c10-4737-4db0-8fa2-19b2e0032959 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:35.192345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30635c10-4737-4db0-8fa2-19b2e0032959 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:35.193594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 659/1332] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:48:35.200079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6aead-e706-4b0c-a33b-f2cdc0795622 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:35.200530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6aead-e706-4b0c-a33b-f2cdc0795622 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:35.200632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6aead-e706-4b0c-a33b-f2cdc0795622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:35.265632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:35.268278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-36c6aead-e706-4b0c-a33b-f2cdc0795622 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:35.269494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-36c6aead-e706-4b0c-a33b-f2cdc0795622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:35.374175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 674/1333] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:35.666679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30635c10-4737-4db0-8fa2-19b2e0032959 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:35.668296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-30635c10-4737-4db0-8fa2-19b2e0032959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:35.774200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30635c10-4737-4db0-8fa2-19b2e0032959 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:35.848658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0b07a-7e2b-4fe9-a348-b87035712535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:35.849258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0b07a-7e2b-4fe9-a348-b87035712535 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:35.849529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0b07a-7e2b-4fe9-a348-b87035712535 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:35.849788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0b07a-7e2b-4fe9-a348-b87035712535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:35.851412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-95c0b07a-7e2b-4fe9-a348-b87035712535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:35.878407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:35.960457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0b07a-7e2b-4fe9-a348-b87035712535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:35.962267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0b07a-7e2b-4fe9-a348-b87035712535 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:35.980477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 660/1334] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:38.001161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977841dd-b2d5-4390-827d-b65af039db26 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:38.001727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977841dd-b2d5-4390-827d-b65af039db26 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:38.001727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977841dd-b2d5-4390-827d-b65af039db26 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:38.003071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 675/1335] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:38.008496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41961726-b864-4ff8-b985-1c682391d9b2 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:38.008959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41961726-b864-4ff8-b985-1c682391d9b2 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:38.009055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41961726-b864-4ff8-b985-1c682391d9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:38.076549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-41961726-b864-4ff8-b985-1c682391d9b2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:38.077919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-41961726-b864-4ff8-b985-1c682391d9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:38.154528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:38.176951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 661/1336] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:38.462385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977841dd-b2d5-4390-827d-b65af039db26 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:38.463977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-977841dd-b2d5-4390-827d-b65af039db26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:38.569489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977841dd-b2d5-4390-827d-b65af039db26 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:38.598637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:38.645528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67a1ad-ea12-41ba-b746-f87af0d768e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:38.646137 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67a1ad-ea12-41ba-b746-f87af0d768e1 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:38.646416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67a1ad-ea12-41ba-b746-f87af0d768e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:38.646687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67a1ad-ea12-41ba-b746-f87af0d768e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:38.648354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9f67a1ad-ea12-41ba-b746-f87af0d768e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:38.758461 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67a1ad-ea12-41ba-b746-f87af0d768e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:38.759940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67a1ad-ea12-41ba-b746-f87af0d768e1 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:38.780018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 676/1337] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:41.368366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154b31b-59b3-4a26-b074-0961b9cd5a5d None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:41.369175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154b31b-59b3-4a26-b074-0961b9cd5a5d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:41.369175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154b31b-59b3-4a26-b074-0961b9cd5a5d None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:41.370199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 662/1338] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:41.375756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e681e-06dd-443a-adb5-4235216237b0 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:41.376215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e681e-06dd-443a-adb5-4235216237b0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:41.376310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e681e-06dd-443a-adb5-4235216237b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:41.447583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ce0e681e-06dd-443a-adb5-4235216237b0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:41.448804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ce0e681e-06dd-443a-adb5-4235216237b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:41.457082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:41.550515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 677/1339] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:41.851698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154b31b-59b3-4a26-b074-0961b9cd5a5d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:41.853372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b154b31b-59b3-4a26-b074-0961b9cd5a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:41.961189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154b31b-59b3-4a26-b074-0961b9cd5a5d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:42.041655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c37f7-164d-4391-8094-e33626440213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:42.042267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c37f7-164d-4391-8094-e33626440213 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:42.042580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c37f7-164d-4391-8094-e33626440213 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:42.042851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c37f7-164d-4391-8094-e33626440213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:42.044556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1a0c37f7-164d-4391-8094-e33626440213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:42.087402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:42.169341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c37f7-164d-4391-8094-e33626440213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:42.170785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c37f7-164d-4391-8094-e33626440213 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:42.190224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 663/1340] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:46.435379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841babc-e712-4ac0-853a-28b18875834f None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:46.435946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841babc-e712-4ac0-853a-28b18875834f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:46.436039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841babc-e712-4ac0-853a-28b18875834f None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:46.437391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 678/1341] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:46.442789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35968b3-4dff-40d2-9d02-62905ff441f0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:46.443289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35968b3-4dff-40d2-9d02-62905ff441f0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:46.443381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35968b3-4dff-40d2-9d02-62905ff441f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:46.516262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f35968b3-4dff-40d2-9d02-62905ff441f0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:46.517471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f35968b3-4dff-40d2-9d02-62905ff441f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:46.612486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:46.624053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 664/1342] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:46.917986 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841babc-e712-4ac0-853a-28b18875834f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:46.919598 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6841babc-e712-4ac0-853a-28b18875834f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:47.030077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841babc-e712-4ac0-853a-28b18875834f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:47.067842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:47.106472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5f3b6-8a61-4e24-bcc1-908f4cdcc1a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:47.107611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f3b6-8a61-4e24-bcc1-908f4cdcc1a1 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:47.108124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f3b6-8a61-4e24-bcc1-908f4cdcc1a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:47.108668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5f3b6-8a61-4e24-bcc1-908f4cdcc1a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:47.110756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2fd5f3b6-8a61-4e24-bcc1-908f4cdcc1a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:47.238048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5f3b6-8a61-4e24-bcc1-908f4cdcc1a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:47.239563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5f3b6-8a61-4e24-bcc1-908f4cdcc1a1 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:47.260350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 679/1343] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:50.223152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c41d8-f1a0-44a2-bee7-a0e457d2cc47 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:50.223666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c41d8-f1a0-44a2-bee7-a0e457d2cc47 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:50.223666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c41d8-f1a0-44a2-bee7-a0e457d2cc47 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:50.225098 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 665/1344] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:50.230254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2a7e8-8193-4d0a-ae48-e9d8c1d15ec6 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:50.230665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2a7e8-8193-4d0a-ae48-e9d8c1d15ec6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:50.230889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2a7e8-8193-4d0a-ae48-e9d8c1d15ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:50.304110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-45f2a7e8-8193-4d0a-ae48-e9d8c1d15ec6 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:50.305375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-45f2a7e8-8193-4d0a-ae48-e9d8c1d15ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:50.324971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:50.411328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 680/1345] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:50.703734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c41d8-f1a0-44a2-bee7-a0e457d2cc47 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:50.705347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9d8c41d8-f1a0-44a2-bee7-a0e457d2cc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:50.821117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c41d8-f1a0-44a2-bee7-a0e457d2cc47 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:50.903189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b82381-bcd0-419d-bb4c-31c60bde2c68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:50.903953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82381-bcd0-419d-bb4c-31c60bde2c68 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:50.904257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82381-bcd0-419d-bb4c-31c60bde2c68 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:50.904527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82381-bcd0-419d-bb4c-31c60bde2c68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:50.906264 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-56b82381-bcd0-419d-bb4c-31c60bde2c68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:50.955985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:51.025662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b82381-bcd0-419d-bb4c-31c60bde2c68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:51.027109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b82381-bcd0-419d-bb4c-31c60bde2c68 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:51.053386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 666/1346] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:48:55.569375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cf331-2b02-4121-afe0-e6ebd7335332 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:55.570708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cf331-2b02-4121-afe0-e6ebd7335332 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:55.570862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cf331-2b02-4121-afe0-e6ebd7335332 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:55.572673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 681/1347] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:48:55.578516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331aa1c-df3e-4e84-8313-afab8a5a693c None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:55.579013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331aa1c-df3e-4e84-8313-afab8a5a693c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:55.579136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331aa1c-df3e-4e84-8313-afab8a5a693c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:55.664207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5331aa1c-df3e-4e84-8313-afab8a5a693c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:48:55.665467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5331aa1c-df3e-4e84-8313-afab8a5a693c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:55.775442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:55.778027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 667/1348] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:48:56.065081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cf331-2b02-4121-afe0-e6ebd7335332 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:48:56.066696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ac5cf331-2b02-4121-afe0-e6ebd7335332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:56.193531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cf331-2b02-4121-afe0-e6ebd7335332 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:48:56.244002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:48:56.269673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8134c57-21c9-408f-8467-812f247f79f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:48:56.270278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134c57-21c9-408f-8467-812f247f79f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:48:56.270553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134c57-21c9-408f-8467-812f247f79f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:48:56.270974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8134c57-21c9-408f-8467-812f247f79f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:48:56.272860 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e8134c57-21c9-408f-8467-812f247f79f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:48:56.385620 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8134c57-21c9-408f-8467-812f247f79f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:48:56.387094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8134c57-21c9-408f-8467-812f247f79f5 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:48:56.405499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 682/1349] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:01.467022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066cc6f-2741-4714-bb9b-3ed56a699218 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:01.467852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066cc6f-2741-4714-bb9b-3ed56a699218 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:01.468738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066cc6f-2741-4714-bb9b-3ed56a699218 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:01.470271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 668/1350] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 15:49:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:49:01.476252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f8dd0-662b-46c4-94dd-cc01fc3553f3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:01.477031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f8dd0-662b-46c4-94dd-cc01fc3553f3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:01.477480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f8dd0-662b-46c4-94dd-cc01fc3553f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:01.621930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ff3f8dd0-662b-46c4-94dd-cc01fc3553f3 None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:01.623156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ff3f8dd0-662b-46c4-94dd-cc01fc3553f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:01.643889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:01.749421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 683/1351] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:06.127324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb29d-de5c-4c55-a0ec-43210b897fbc None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:06.127962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb29d-de5c-4c55-a0ec-43210b897fbc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:06.128141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb29d-de5c-4c55-a0ec-43210b897fbc None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:06.130220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 669/1352] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 15:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:06.136281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad48e1-19ea-45d3-984c-c50e09802e01 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:06.136771 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad48e1-19ea-45d3-984c-c50e09802e01 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:06.136854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad48e1-19ea-45d3-984c-c50e09802e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:06.220582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-74ad48e1-19ea-45d3-984c-c50e09802e01 None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:06.221886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-74ad48e1-19ea-45d3-984c-c50e09802e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:06.301675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:06.341786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 684/1353] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 15:49:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:07.624418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491806c-33cf-4ab7-a79a-8a9081cd8d75 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:07.624995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491806c-33cf-4ab7-a79a-8a9081cd8d75 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:07.624995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491806c-33cf-4ab7-a79a-8a9081cd8d75 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:07.626273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 670/1354] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 15:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:07.631333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae1b8-c8d1-4e1d-9d34-e8a79c64eeb3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:07.632461 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae1b8-c8d1-4e1d-9d34-e8a79c64eeb3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:07.632544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ae1b8-c8d1-4e1d-9d34-e8a79c64eeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:07.713222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-042ae1b8-c8d1-4e1d-9d34-e8a79c64eeb3 None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:07.714701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-042ae1b8-c8d1-4e1d-9d34-e8a79c64eeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:07.829008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 685/1355] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 15:49:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:11.296982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b108d-258e-4d8f-8c4f-b56a64728694 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:11.297527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b108d-258e-4d8f-8c4f-b56a64728694 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:11.297527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b108d-258e-4d8f-8c4f-b56a64728694 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:11.298854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 671/1356] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:11.305001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9560c7-bb17-402f-879b-a7be99d6854d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:11.305552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9560c7-bb17-402f-879b-a7be99d6854d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:11.305552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9560c7-bb17-402f-879b-a7be99d6854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:11.343753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:11.389197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8d9560c7-bb17-402f-879b-a7be99d6854d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:11.390566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8d9560c7-bb17-402f-879b-a7be99d6854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:11.514806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 686/1357] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:11.824650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b814bd-4569-4e08-b425-fd7c88069c81 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:11.824952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b814bd-4569-4e08-b425-fd7c88069c81 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:11.825188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b814bd-4569-4e08-b425-fd7c88069c81 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:11.827436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 672/1358] 10.222.0.94 () {54 vars in 962 bytes} [Fri Aug 26 15:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:11.837473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14b458-31e1-46e3-9815-e6f4d98450be None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:11.838218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14b458-31e1-46e3-9815-e6f4d98450be None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:11.838827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14b458-31e1-46e3-9815-e6f4d98450be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:11.933357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-df14b458-31e1-46e3-9815-e6f4d98450be None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:11.934635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-df14b458-31e1-46e3-9815-e6f4d98450be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:11.996091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:12.091795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 687/1359] 10.222.0.94 () {58 vars in 1059 bytes} [Fri Aug 26 15:49:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:49:12.103666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b814bd-4569-4e08-b425-fd7c88069c81 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:12.106806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-76b814bd-4569-4e08-b425-fd7c88069c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:12.359679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b814bd-4569-4e08-b425-fd7c88069c81 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:12.480604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:12.536929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f3a53-7e21-4a83-872f-f4c2f5e529f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:12.538368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f3a53-7e21-4a83-872f-f4c2f5e529f7 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:12.538805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f3a53-7e21-4a83-872f-f4c2f5e529f7 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:12.539200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f3a53-7e21-4a83-872f-f4c2f5e529f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:12.540983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b43f3a53-7e21-4a83-872f-f4c2f5e529f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:12.702247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f3a53-7e21-4a83-872f-f4c2f5e529f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:12.703865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f3a53-7e21-4a83-872f-f4c2f5e529f7 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:12.728422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 673/1360] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:49:15.017160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68746b20-491f-42b9-a17c-dcf3e074fab0 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:15.017621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68746b20-491f-42b9-a17c-dcf3e074fab0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:15.017936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68746b20-491f-42b9-a17c-dcf3e074fab0 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:15.019424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 688/1361] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:15.027005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a681-ab33-48d9-9b85-69cdad2161d0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:15.027531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a681-ab33-48d9-9b85-69cdad2161d0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:15.027756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a681-ab33-48d9-9b85-69cdad2161d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:15.112381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0c37a681-ab33-48d9-9b85-69cdad2161d0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:15.113742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0c37a681-ab33-48d9-9b85-69cdad2161d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:15.164345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:15.241860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 674/1362] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:15.577271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68746b20-491f-42b9-a17c-dcf3e074fab0 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:15.577743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-68746b20-491f-42b9-a17c-dcf3e074fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:15.770162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68746b20-491f-42b9-a17c-dcf3e074fab0 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:15.781781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:15.903870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ea54-3070-4836-96a4-00303de933e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:15.904632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ea54-3070-4836-96a4-00303de933e2 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:15.904893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ea54-3070-4836-96a4-00303de933e2 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:15.905280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ea54-3070-4836-96a4-00303de933e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:15.906908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e399ea54-3070-4836-96a4-00303de933e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:16.025265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399ea54-3070-4836-96a4-00303de933e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:16.028079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399ea54-3070-4836-96a4-00303de933e2 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:16.050535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 689/1363] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:49:18.560350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac115f-53a5-438b-9658-3c58c0460a05 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:18.561415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac115f-53a5-438b-9658-3c58c0460a05 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:18.561905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac115f-53a5-438b-9658-3c58c0460a05 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:18.566404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 675/1364] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:18.570015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20704cc-ee8a-4461-97f1-c211f1819417 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:18.570798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20704cc-ee8a-4461-97f1-c211f1819417 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:18.571244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20704cc-ee8a-4461-97f1-c211f1819417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:18.588841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:18.657535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d20704cc-ee8a-4461-97f1-c211f1819417 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:18.658758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d20704cc-ee8a-4461-97f1-c211f1819417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:18.786038 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 690/1365] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:19.087017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acf370-e4a8-4653-b6ed-d69ddc72fbfc None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:19.087347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acf370-e4a8-4653-b6ed-d69ddc72fbfc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:19.087569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acf370-e4a8-4653-b6ed-d69ddc72fbfc None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:19.089113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 676/1366] 10.222.0.94 () {54 vars in 962 bytes} [Fri Aug 26 15:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:19.095202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea8ea0-6974-42f2-9d49-cd25d13181ca None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:19.096886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea8ea0-6974-42f2-9d49-cd25d13181ca None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:19.096886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea8ea0-6974-42f2-9d49-cd25d13181ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:19.197886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-f5ea8ea0-6974-42f2-9d49-cd25d13181ca None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:19.199408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f5ea8ea0-6974-42f2-9d49-cd25d13181ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:19.243401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:19.356633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 691/1367] 10.222.0.94 () {58 vars in 1059 bytes} [Fri Aug 26 15:49:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:19.366528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acf370-e4a8-4653-b6ed-d69ddc72fbfc None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:19.368588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-62acf370-e4a8-4653-b6ed-d69ddc72fbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:19.589069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acf370-e4a8-4653-b6ed-d69ddc72fbfc None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:19.612560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:19.724158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4f602-73fa-4041-894b-797535f61d38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:19.724804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4f602-73fa-4041-894b-797535f61d38 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:19.725145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4f602-73fa-4041-894b-797535f61d38 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:19.725410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4f602-73fa-4041-894b-797535f61d38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:19.727147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-05e4f602-73fa-4041-894b-797535f61d38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:19.836002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4f602-73fa-4041-894b-797535f61d38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:19.839303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4f602-73fa-4041-894b-797535f61d38 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:19.879536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 677/1368] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:20.297676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b50277-8187-4d1a-abbf-92652d28d1be None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:20.298264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b50277-8187-4d1a-abbf-92652d28d1be None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:20.298264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b50277-8187-4d1a-abbf-92652d28d1be None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:20.300008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 692/1369] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:20.306173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebbbac-80b4-45e3-95e5-3a739ff64dc9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:20.306528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebbbac-80b4-45e3-95e5-3a739ff64dc9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:20.306723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebbbac-80b4-45e3-95e5-3a739ff64dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:20.351642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:20.399218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-30ebbbac-80b4-45e3-95e5-3a739ff64dc9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:20.400745 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-30ebbbac-80b4-45e3-95e5-3a739ff64dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:20.522288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 678/1370] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:21.139063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98aeb-7b1f-4e13-b13c-b76e66735c32 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:21.139063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98aeb-7b1f-4e13-b13c-b76e66735c32 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:21.139063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98aeb-7b1f-4e13-b13c-b76e66735c32 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:21.141240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 693/1371] 10.222.0.94 () {54 vars in 964 bytes} [Fri Aug 26 15:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:21.145889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e788f6-88b3-48a5-922a-e8c0cc092642 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:21.145889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e788f6-88b3-48a5-922a-e8c0cc092642 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:21.145889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e788f6-88b3-48a5-922a-e8c0cc092642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:21.234183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-80e788f6-88b3-48a5-922a-e8c0cc092642 None None] MFA Rules not processed for user `58567e28a5f741eda021aecab43d6a63`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:21.235473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-80e788f6-88b3-48a5-922a-e8c0cc092642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:21.294246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:21.331701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 679/1372] 10.222.0.94 () {58 vars in 1061 bytes} [Fri Aug 26 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:21.340106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98aeb-7b1f-4e13-b13c-b76e66735c32 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:21.341614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d9a98aeb-7b1f-4e13-b13c-b76e66735c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:21.471532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98aeb-7b1f-4e13-b13c-b76e66735c32 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:21.516942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:21.552455 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c66005-9ddc-4959-bb0a-296c658e6dbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:21.553622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c66005-9ddc-4959-bb0a-296c658e6dbc service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:21.553893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c66005-9ddc-4959-bb0a-296c658e6dbc service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:21.554159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c66005-9ddc-4959-bb0a-296c658e6dbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:21.555807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a5c66005-9ddc-4959-bb0a-296c658e6dbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:21.696160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c66005-9ddc-4959-bb0a-296c658e6dbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:21.697667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c66005-9ddc-4959-bb0a-296c658e6dbc service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:21.728956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 694/1373] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:49:22.142847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a1d94-5458-4748-a626-307338c728da None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:22.143162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a1d94-5458-4748-a626-307338c728da None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:22.143442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a1d94-5458-4748-a626-307338c728da None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:22.145580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 680/1374] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:22.151180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13d04-bf02-44fd-b5e1-9ed93cc8e582 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:22.151472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13d04-bf02-44fd-b5e1-9ed93cc8e582 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:22.151749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b13d04-bf02-44fd-b5e1-9ed93cc8e582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:22.220214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-51b13d04-bf02-44fd-b5e1-9ed93cc8e582 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:22.221434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-51b13d04-bf02-44fd-b5e1-9ed93cc8e582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:22.243602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:22.323424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 695/1375] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:22.620181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a1d94-5458-4748-a626-307338c728da None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:22.621755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1e3a1d94-5458-4748-a626-307338c728da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:22.779650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a1d94-5458-4748-a626-307338c728da None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:22.831542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:22.901556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b61b9a-b721-4ebf-bfd4-fdea30366052 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:22.902172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b61b9a-b721-4ebf-bfd4-fdea30366052 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:22.904102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b61b9a-b721-4ebf-bfd4-fdea30366052 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:22.904390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b61b9a-b721-4ebf-bfd4-fdea30366052 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:22.906911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-38b61b9a-b721-4ebf-bfd4-fdea30366052 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:23.007378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b61b9a-b721-4ebf-bfd4-fdea30366052 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:23.009034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b61b9a-b721-4ebf-bfd4-fdea30366052 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:23.027773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 681/1376] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:49:25.148372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a547d-02c2-49ab-81a4-a41ae662c2fa None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:25.148949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a547d-02c2-49ab-81a4-a41ae662c2fa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:25.148949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a547d-02c2-49ab-81a4-a41ae662c2fa None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:25.150291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 696/1377] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:25.156030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2eb3-09ab-4444-97e5-e596dc2ce945 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:25.158320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2eb3-09ab-4444-97e5-e596dc2ce945 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:25.158320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2eb3-09ab-4444-97e5-e596dc2ce945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:25.227329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-1c9c2eb3-09ab-4444-97e5-e596dc2ce945 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:25.228458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1c9c2eb3-09ab-4444-97e5-e596dc2ce945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:25.235390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:25.335948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 682/1378] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:25.644492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a547d-02c2-49ab-81a4-a41ae662c2fa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:25.647335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a40a547d-02c2-49ab-81a4-a41ae662c2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:25.807142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a547d-02c2-49ab-81a4-a41ae662c2fa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:25.811174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:25.932033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d86cb-1126-4128-b7d8-bef77d383b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:25.932656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d86cb-1126-4128-b7d8-bef77d383b35 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:25.932934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d86cb-1126-4128-b7d8-bef77d383b35 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:25.933202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d86cb-1126-4128-b7d8-bef77d383b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:25.935107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ee7d86cb-1126-4128-b7d8-bef77d383b35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:26.036413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d86cb-1126-4128-b7d8-bef77d383b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:26.038604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d86cb-1126-4128-b7d8-bef77d383b35 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:26.059471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 697/1379] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:28.192632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8df8a7-0797-40f2-a271-fa79da67f7b4 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:28.193716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8df8a7-0797-40f2-a271-fa79da67f7b4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:28.193716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8df8a7-0797-40f2-a271-fa79da67f7b4 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:28.195670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 683/1380] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:49:28.201100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b027b4-d293-4295-b0dc-84e3f890b9bf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:28.201622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b027b4-d293-4295-b0dc-84e3f890b9bf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:28.201622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b027b4-d293-4295-b0dc-84e3f890b9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:28.214205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:28.274911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-93b027b4-d293-4295-b0dc-84e3f890b9bf None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:28.276168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-93b027b4-d293-4295-b0dc-84e3f890b9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:28.385175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 698/1381] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:28.686738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8df8a7-0797-40f2-a271-fa79da67f7b4 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:28.689263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ee8df8a7-0797-40f2-a271-fa79da67f7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:28.848622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:28.860541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8df8a7-0797-40f2-a271-fa79da67f7b4 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:28.991102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc6420-4232-4d46-8079-64912f2deaa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:28.991708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc6420-4232-4d46-8079-64912f2deaa6 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:28.991984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc6420-4232-4d46-8079-64912f2deaa6 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:28.992248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc6420-4232-4d46-8079-64912f2deaa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:28.993853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-dcbc6420-4232-4d46-8079-64912f2deaa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:29.104039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc6420-4232-4d46-8079-64912f2deaa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:29.106390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc6420-4232-4d46-8079-64912f2deaa6 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:29.118951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:29.127436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 684/1382] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:31.287722 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8e4be-89d3-45f2-b23b-d9fb7ce3919c None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:31.288306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8e4be-89d3-45f2-b23b-d9fb7ce3919c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:31.288525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8e4be-89d3-45f2-b23b-d9fb7ce3919c None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:31.290484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 699/1383] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:31.296287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bb7ba-a25e-440f-ba1e-5509b3b1e7c0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:31.296813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bb7ba-a25e-440f-ba1e-5509b3b1e7c0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:31.296813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bb7ba-a25e-440f-ba1e-5509b3b1e7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:31.378386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-1c6bb7ba-a25e-440f-ba1e-5509b3b1e7c0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:31.379986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1c6bb7ba-a25e-440f-ba1e-5509b3b1e7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:31.514056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 685/1384] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:31.809374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8e4be-89d3-45f2-b23b-d9fb7ce3919c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:31.811021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d3f8e4be-89d3-45f2-b23b-d9fb7ce3919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:31.914417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:31.985574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8e4be-89d3-45f2-b23b-d9fb7ce3919c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:32.117609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25384f9-3c2a-471f-a1f2-dfddf731c98c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:32.118248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25384f9-3c2a-471f-a1f2-dfddf731c98c service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:32.118539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25384f9-3c2a-471f-a1f2-dfddf731c98c service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:32.118845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25384f9-3c2a-471f-a1f2-dfddf731c98c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:32.120561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a25384f9-3c2a-471f-a1f2-dfddf731c98c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:32.176292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:32.220028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25384f9-3c2a-471f-a1f2-dfddf731c98c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:32.221726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25384f9-3c2a-471f-a1f2-dfddf731c98c service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:32.241689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 700/1385] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:34.425883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b105-21c1-4977-8bee-757ff5b17a62 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:34.426486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b105-21c1-4977-8bee-757ff5b17a62 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:34.426648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b105-21c1-4977-8bee-757ff5b17a62 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:34.429246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 686/1386] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:34.436190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75339526-ee0a-44d3-9260-3975f2abe5a8 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:34.436746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75339526-ee0a-44d3-9260-3975f2abe5a8 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:34.436746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75339526-ee0a-44d3-9260-3975f2abe5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:34.513618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-75339526-ee0a-44d3-9260-3975f2abe5a8 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:34.515579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-75339526-ee0a-44d3-9260-3975f2abe5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:34.638843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 701/1387] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:49:34.940976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b105-21c1-4977-8bee-757ff5b17a62 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:34.942490 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9178b105-21c1-4977-8bee-757ff5b17a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:35.025855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:35.101948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b105-21c1-4977-8bee-757ff5b17a62 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:35.244960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f81ade-cb6f-45f3-8820-147d37dc2eec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:35.245480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81ade-cb6f-45f3-8820-147d37dc2eec service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:35.245751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81ade-cb6f-45f3-8820-147d37dc2eec service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:35.246014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81ade-cb6f-45f3-8820-147d37dc2eec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:35.247776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e0f81ade-cb6f-45f3-8820-147d37dc2eec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:35.300012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:35.370219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f81ade-cb6f-45f3-8820-147d37dc2eec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:35.372212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f81ade-cb6f-45f3-8820-147d37dc2eec service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:35.393303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 687/1388] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:37.826792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae630268-052e-4661-a591-f2942df104b7 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:37.827435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae630268-052e-4661-a591-f2942df104b7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:37.827435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae630268-052e-4661-a591-f2942df104b7 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:37.830952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 702/1389] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:37.837246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a01aea-6098-452e-83ee-6f76cf06f80c None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:37.837924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a01aea-6098-452e-83ee-6f76cf06f80c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:37.837924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a01aea-6098-452e-83ee-6f76cf06f80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:37.916321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-88a01aea-6098-452e-83ee-6f76cf06f80c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:37.917547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-88a01aea-6098-452e-83ee-6f76cf06f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:38.038252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 688/1390] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:49:38.364764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae630268-052e-4661-a591-f2942df104b7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:38.364764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ae630268-052e-4661-a591-f2942df104b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:38.406069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:38.560588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae630268-052e-4661-a591-f2942df104b7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:38.675171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:38.688237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088fc3c-92ce-4a30-992b-7b061eea035d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:38.694141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088fc3c-92ce-4a30-992b-7b061eea035d service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:38.694141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088fc3c-92ce-4a30-992b-7b061eea035d service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:38.694141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088fc3c-92ce-4a30-992b-7b061eea035d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:38.694141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9088fc3c-92ce-4a30-992b-7b061eea035d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:38.806659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088fc3c-92ce-4a30-992b-7b061eea035d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:38.809570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088fc3c-92ce-4a30-992b-7b061eea035d service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:38.847327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 703/1391] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:38.877322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e48ce0-bb4a-4fda-9d68-3b359cc6be23 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:38.877902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e48ce0-bb4a-4fda-9d68-3b359cc6be23 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:38.877902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e48ce0-bb4a-4fda-9d68-3b359cc6be23 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:38.879253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 689/1392] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:38.885871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5afd14-764f-46a7-961c-0e63f375a64c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:38.886127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5afd14-764f-46a7-961c-0e63f375a64c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:38.886379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5afd14-764f-46a7-961c-0e63f375a64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:38.961252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4e5afd14-764f-46a7-961c-0e63f375a64c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:38.962531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4e5afd14-764f-46a7-961c-0e63f375a64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:39.025567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:39.072288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 704/1393] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:39.376985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e0f0e-4903-4301-bf75-3e2c6928f91c None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:39.377258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e0f0e-4903-4301-bf75-3e2c6928f91c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:39.377522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e0f0e-4903-4301-bf75-3e2c6928f91c None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:39.378835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 690/1394] 10.222.0.94 () {54 vars in 967 bytes} [Fri Aug 26 15:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:39.384528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f265-0e14-441d-8980-7146ae671a68 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:39.384808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f265-0e14-441d-8980-7146ae671a68 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:39.385077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f265-0e14-441d-8980-7146ae671a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:39.467642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-dfb6f265-0e14-441d-8980-7146ae671a68 None None] MFA Rules not processed for user `23ddf8f600454da7ae9b715cd4cbd186`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:39.468864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-dfb6f265-0e14-441d-8980-7146ae671a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:39.549038 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 705/1395] 10.222.0.94 () {58 vars in 1064 bytes} [Fri Aug 26 15:49:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:39.557050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e0f0e-4903-4301-bf75-3e2c6928f91c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:39.558694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ed0e0f0e-4903-4301-bf75-3e2c6928f91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:39.574249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:39.686058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e0f0e-4903-4301-bf75-3e2c6928f91c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:39.769328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517343c2-b6f9-402e-85be-4931f322ba9d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23ddf8f600454da7ae9b715cd4cbd186', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:39.769924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517343c2-b6f9-402e-85be-4931f322ba9d service glance] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:39.770223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517343c2-b6f9-402e-85be-4931f322ba9d service glance] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:39.770489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517343c2-b6f9-402e-85be-4931f322ba9d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:39.773075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-517343c2-b6f9-402e-85be-4931f322ba9d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:39.854943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:39.901802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517343c2-b6f9-402e-85be-4931f322ba9d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:39.903319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517343c2-b6f9-402e-85be-4931f322ba9d service glance] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:39.923440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 691/1396] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:40.809976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad6c05-d13c-4471-87f3-ba22c00ed049 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:40.810521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad6c05-d13c-4471-87f3-ba22c00ed049 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:40.810521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad6c05-d13c-4471-87f3-ba22c00ed049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:40.887628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a4ad6c05-d13c-4471-87f3-ba22c00ed049 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:40.888854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a4ad6c05-d13c-4471-87f3-ba22c00ed049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:40.899789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:40.982711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 706/1397] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 15:49:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:41.016651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b7345-7231-49a0-bd77-fca1d47c1555 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:41.017072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b7345-7231-49a0-bd77-fca1d47c1555 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:41.017488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b7345-7231-49a0-bd77-fca1d47c1555 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:41.018516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 692/1398] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:41.025101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03998d-7bca-450e-9df8-b565f0729b5e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:41.025388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03998d-7bca-450e-9df8-b565f0729b5e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:41.025659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03998d-7bca-450e-9df8-b565f0729b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:41.030402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2900d5e-5af9-4b0b-aea0-3e81494175e2 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:41.030678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2900d5e-5af9-4b0b-aea0-3e81494175e2 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:41.031051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2900d5e-5af9-4b0b-aea0-3e81494175e2 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:41.032522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 693/1399] 10.222.0.94 () {54 vars in 943 bytes} [Fri Aug 26 15:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:49:41.039013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadd6c2-040c-4b67-b691-be600b913504 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:41.039293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadd6c2-040c-4b67-b691-be600b913504 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:41.039563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadd6c2-040c-4b67-b691-be600b913504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:41.095896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4c03998d-7bca-450e-9df8-b565f0729b5e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:41.097000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4c03998d-7bca-450e-9df8-b565f0729b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:41.118547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-9dadd6c2-040c-4b67-b691-be600b913504 None None] MFA Rules not processed for user `b92a2d92e27d4fe48b42f9e04be92372`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:41.119796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9dadd6c2-040c-4b67-b691-be600b913504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:41.201680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 707/1400] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:41.235168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 694/1401] 10.222.0.94 () {58 vars in 1040 bytes} [Fri Aug 26 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:49:41.245552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03998d-7bca-450e-9df8-b565f0729b5e None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:41.246784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4c03998d-7bca-450e-9df8-b565f0729b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:41.249421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:41.355155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03998d-7bca-450e-9df8-b565f0729b5e None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:41.462416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afa0f1-fe32-47a8-b72c-d804c8deb5a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:41.463064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afa0f1-fe32-47a8-b72c-d804c8deb5a1 service swift] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:41.463336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afa0f1-fe32-47a8-b72c-d804c8deb5a1 service swift] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:41.463596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afa0f1-fe32-47a8-b72c-d804c8deb5a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:41.465391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-94afa0f1-fe32-47a8-b72c-d804c8deb5a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:41.488817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:41.503302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadd6c2-040c-4b67-b691-be600b913504 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:41.504524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9dadd6c2-040c-4b67-b691-be600b913504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:41.508300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:41.559239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afa0f1-fe32-47a8-b72c-d804c8deb5a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:41.561594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afa0f1-fe32-47a8-b72c-d804c8deb5a1 service swift] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:41.571331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 708/1402] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 15:49:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:49:41.615598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadd6c2-040c-4b67-b691-be600b913504 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:41.737283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d09169-540c-4946-bb9a-f3ef9df0f60a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:41.737927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09169-540c-4946-bb9a-f3ef9df0f60a service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:41.738357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09169-540c-4946-bb9a-f3ef9df0f60a service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:41.738810 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09169-540c-4946-bb9a-f3ef9df0f60a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:41.741289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-90d09169-540c-4946-bb9a-f3ef9df0f60a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:41.754582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:41.842694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d09169-540c-4946-bb9a-f3ef9df0f60a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:41.844343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d09169-540c-4946-bb9a-f3ef9df0f60a service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:41.872462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 695/1403] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:49:44.268146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364ed6c-6fdd-40b8-b785-fd2ab3025c53 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:44.268690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364ed6c-6fdd-40b8-b785-fd2ab3025c53 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:44.268690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364ed6c-6fdd-40b8-b785-fd2ab3025c53 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:44.275152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 709/1404] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:44.276574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db81ee-196d-49fb-9e0c-da3c1313accf None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:44.277039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db81ee-196d-49fb-9e0c-da3c1313accf None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:44.277134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db81ee-196d-49fb-9e0c-da3c1313accf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:44.358037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-63db81ee-196d-49fb-9e0c-da3c1313accf None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:44.359345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-63db81ee-196d-49fb-9e0c-da3c1313accf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:44.478906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:44.492943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 696/1405] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:44.790512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364ed6c-6fdd-40b8-b785-fd2ab3025c53 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:44.792088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9364ed6c-6fdd-40b8-b785-fd2ab3025c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:44.972572 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364ed6c-6fdd-40b8-b785-fd2ab3025c53 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:44.991926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:45.105746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc7425-581f-4530-9d39-fa21697612bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:45.106373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc7425-581f-4530-9d39-fa21697612bf service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:45.106778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc7425-581f-4530-9d39-fa21697612bf service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:45.107079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc7425-581f-4530-9d39-fa21697612bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:45.108911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a8bc7425-581f-4530-9d39-fa21697612bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:45.213736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc7425-581f-4530-9d39-fa21697612bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:45.215390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc7425-581f-4530-9d39-fa21697612bf service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:45.235280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 710/1406] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:47.375106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50467-cfee-4d40-a402-d0779732cc57 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:47.375663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50467-cfee-4d40-a402-d0779732cc57 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:47.375663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50467-cfee-4d40-a402-d0779732cc57 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:47.377361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 697/1407] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:49:47.383373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4e335-1e45-432b-8727-28872648ea65 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:47.384459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4e335-1e45-432b-8727-28872648ea65 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:47.384955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4e335-1e45-432b-8727-28872648ea65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:47.415807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:47.467927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d0f4e335-1e45-432b-8727-28872648ea65 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:49:47.469185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d0f4e335-1e45-432b-8727-28872648ea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:47.589426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 711/1408] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:49:47.882478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f50467-cfee-4d40-a402-d0779732cc57 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:49:47.884209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-21f50467-cfee-4d40-a402-d0779732cc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:48.060798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f50467-cfee-4d40-a402-d0779732cc57 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:49:48.140065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:48.193913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15104409-339d-449d-9fed-3d05398dab5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:49:48.194659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15104409-339d-449d-9fed-3d05398dab5e service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:49:48.195080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15104409-339d-449d-9fed-3d05398dab5e service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:49:48.195436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15104409-339d-449d-9fed-3d05398dab5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:49:48.197145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-15104409-339d-449d-9fed-3d05398dab5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:49:48.306109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15104409-339d-449d-9fed-3d05398dab5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:49:48.307768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15104409-339d-449d-9fed-3d05398dab5e service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:49:48.332107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 698/1409] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:49:57.587117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=56 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:49:58.329591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=66 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:05.550264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948eeb-76a2-48b3-91b4-53ce8072facd None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:05.550814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948eeb-76a2-48b3-91b4-53ce8072facd None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:05.550814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948eeb-76a2-48b3-91b4-53ce8072facd None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:05.552288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 712/1410] 10.222.0.94 () {54 vars in 918 bytes} [Fri Aug 26 15:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:05.557484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b548-8a39-4121-b5f8-8e8deb0a8911 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:05.557945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b548-8a39-4121-b5f8-8e8deb0a8911 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:05.558051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b548-8a39-4121-b5f8-8e8deb0a8911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:05.645799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ecd2b548-8a39-4121-b5f8-8e8deb0a8911 None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:05.647034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ecd2b548-8a39-4121-b5f8-8e8deb0a8911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:05.757437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 699/1411] 10.222.0.94 () {58 vars in 1015 bytes} [Fri Aug 26 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:05.771297 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5701d4-454a-4e4f-b39f-233bf0ae6653 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:05.771580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5701d4-454a-4e4f-b39f-233bf0ae6653 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:05.771854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5701d4-454a-4e4f-b39f-233bf0ae6653 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:05.773218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 713/1412] 10.222.0.94 () {54 vars in 918 bytes} [Fri Aug 26 15:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:05.778298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaedde1-3796-49a6-893a-b3596218bd34 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:05.778581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaedde1-3796-49a6-893a-b3596218bd34 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:05.778854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaedde1-3796-49a6-893a-b3596218bd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:05.859155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:05.862463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-dfaedde1-3796-49a6-893a-b3596218bd34 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:05.863762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-dfaedde1-3796-49a6-893a-b3596218bd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:05.996786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 700/1413] 10.222.0.94 () {58 vars in 1015 bytes} [Fri Aug 26 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:08.433987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab485-54bb-4781-b051-5f2e5d68fb89 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:08.434556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab485-54bb-4781-b051-5f2e5d68fb89 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:08.434556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab485-54bb-4781-b051-5f2e5d68fb89 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:08.435904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:08.441679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e05844-80ec-434b-aec6-a4f7e762cadf None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:08.442429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e05844-80ec-434b-aec6-a4f7e762cadf None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:08.442429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e05844-80ec-434b-aec6-a4f7e762cadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:08.515307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-82e05844-80ec-434b-aec6-a4f7e762cadf None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:08.516506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-82e05844-80ec-434b-aec6-a4f7e762cadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:08.572480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:08.618817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:08.912431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab485-54bb-4781-b051-5f2e5d68fb89 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:08.914232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-44bab485-54bb-4781-b051-5f2e5d68fb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:09.063399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab485-54bb-4781-b051-5f2e5d68fb89 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:09.138673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:09.179530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3cc519-5171-4435-b8cf-854438809cf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:09.180072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cc519-5171-4435-b8cf-854438809cf6 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:09.180348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cc519-5171-4435-b8cf-854438809cf6 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:09.180617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cc519-5171-4435-b8cf-854438809cf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:09.182241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4f3cc519-5171-4435-b8cf-854438809cf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:09.275977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cc519-5171-4435-b8cf-854438809cf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:09.277597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cc519-5171-4435-b8cf-854438809cf6 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:09.295702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 715/1416] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:10.220606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311375-7ed5-4080-b4a9-01541dce131c None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:10.221178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311375-7ed5-4080-b4a9-01541dce131c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:10.221178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311375-7ed5-4080-b4a9-01541dce131c None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:10.222568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:10.229291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3403-a593-45ea-a84f-2c56e3bef3bf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:10.229766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3403-a593-45ea-a84f-2c56e3bef3bf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:10.229871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e3403-a593-45ea-a84f-2c56e3bef3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:10.306423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3d9e3403-a593-45ea-a84f-2c56e3bef3bf None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:10.307687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3d9e3403-a593-45ea-a84f-2c56e3bef3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:10.335716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:10.406909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:10.559788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75311375-7ed5-4080-b4a9-01541dce131c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:10.561334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-75311375-7ed5-4080-b4a9-01541dce131c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:10.710915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75311375-7ed5-4080-b4a9-01541dce131c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:10.750259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:10.826259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90334856-5a31-4ed6-a4ac-1e54a030f0b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:10.826897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90334856-5a31-4ed6-a4ac-1e54a030f0b3 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:10.827207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90334856-5a31-4ed6-a4ac-1e54a030f0b3 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:10.827481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90334856-5a31-4ed6-a4ac-1e54a030f0b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:10.829101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-90334856-5a31-4ed6-a4ac-1e54a030f0b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:10.919427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90334856-5a31-4ed6-a4ac-1e54a030f0b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:10.921029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90334856-5a31-4ed6-a4ac-1e54a030f0b3 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:10.940088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 703/1419] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:11.823052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae5c7-810d-4492-b825-a0172b024e82 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:11.823617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae5c7-810d-4492-b825-a0172b024e82 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:11.823617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae5c7-810d-4492-b825-a0172b024e82 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:11.824949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:11.829939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899489-e99e-4df7-8688-3a70d6ccb6a0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:11.830381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899489-e99e-4df7-8688-3a70d6ccb6a0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:11.830489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899489-e99e-4df7-8688-3a70d6ccb6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:11.886535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:11.908794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-02899489-e99e-4df7-8688-3a70d6ccb6a0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:11.910018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-02899489-e99e-4df7-8688-3a70d6ccb6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:12.017082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:12.169534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ae5c7-810d-4492-b825-a0172b024e82 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:12.171134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ea0ae5c7-810d-4492-b825-a0172b024e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:12.317928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ae5c7-810d-4492-b825-a0172b024e82 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:12.329962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:12.433237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38aca1-1493-4b55-9db5-de61cb4f40f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:12.433845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38aca1-1493-4b55-9db5-de61cb4f40f1 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:12.434121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38aca1-1493-4b55-9db5-de61cb4f40f1 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:12.434388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38aca1-1493-4b55-9db5-de61cb4f40f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:12.436028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1f38aca1-1493-4b55-9db5-de61cb4f40f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:12.528288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38aca1-1493-4b55-9db5-de61cb4f40f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:12.529886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38aca1-1493-4b55-9db5-de61cb4f40f1 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:12.547989 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 718/1422] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:13.438673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35084f7f-f948-475b-a707-8a062e3ccd22 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.439266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35084f7f-f948-475b-a707-8a062e3ccd22 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.439266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35084f7f-f948-475b-a707-8a062e3ccd22 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.440808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:13.446777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac585b19-eecf-4bd4-b54d-1c20152e47eb None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:13.447292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac585b19-eecf-4bd4-b54d-1c20152e47eb None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:13.447409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac585b19-eecf-4bd4-b54d-1c20152e47eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:13.468393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:13.528629 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ac585b19-eecf-4bd4-b54d-1c20152e47eb None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:13.529905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ac585b19-eecf-4bd4-b54d-1c20152e47eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:13.634418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:13.787766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35084f7f-f948-475b-a707-8a062e3ccd22 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:13.789307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-35084f7f-f948-475b-a707-8a062e3ccd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:13.899001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:13.937631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35084f7f-f948-475b-a707-8a062e3ccd22 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:14.053717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc06f66-f698-4939-ac0b-20e971827f54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:14.054336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06f66-f698-4939-ac0b-20e971827f54 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:14.055141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06f66-f698-4939-ac0b-20e971827f54 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:14.055141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06f66-f698-4939-ac0b-20e971827f54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:14.056551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4bc06f66-f698-4939-ac0b-20e971827f54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:14.144325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:14.158855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc06f66-f698-4939-ac0b-20e971827f54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:14.160720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc06f66-f698-4939-ac0b-20e971827f54 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:14.180675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 706/1425] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:15.064172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589e845-b52b-4513-ae3c-9106d1b0281b None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.064763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589e845-b52b-4513-ae3c-9106d1b0281b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.064763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589e845-b52b-4513-ae3c-9106d1b0281b None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.066050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:15.072071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded622a-1ac0-4656-ae96-bae4370280f4 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.072552 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded622a-1ac0-4656-ae96-bae4370280f4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.072675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded622a-1ac0-4656-ae96-bae4370280f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.150386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-dded622a-1ac0-4656-ae96-bae4370280f4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:15.151747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-dded622a-1ac0-4656-ae96-bae4370280f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:15.253780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:15.406186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589e845-b52b-4513-ae3c-9106d1b0281b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:15.407754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e589e845-b52b-4513-ae3c-9106d1b0281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:15.495431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:15.558746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589e845-b52b-4513-ae3c-9106d1b0281b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:15.672938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ef6ee-43b8-4963-aca6-56c4ced36f5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:15.673553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef6ee-43b8-4963-aca6-56c4ced36f5a service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:15.673828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef6ee-43b8-4963-aca6-56c4ced36f5a service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:15.674095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef6ee-43b8-4963-aca6-56c4ced36f5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:15.675853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b46ef6ee-43b8-4963-aca6-56c4ced36f5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:15.729986 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:15.776211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ef6ee-43b8-4963-aca6-56c4ced36f5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:15.777807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ef6ee-43b8-4963-aca6-56c4ced36f5a service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:15.795807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 721/1428] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:16.666606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533825c7-0d1c-4ded-b47e-cca5ad71d327 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.667187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533825c7-0d1c-4ded-b47e-cca5ad71d327 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.667187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533825c7-0d1c-4ded-b47e-cca5ad71d327 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.668515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:16.673586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d92bb-aa15-4446-aaef-ad8fdd3ce076 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:16.673996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d92bb-aa15-4446-aaef-ad8fdd3ce076 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:16.674119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d92bb-aa15-4446-aaef-ad8fdd3ce076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:16.870340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d27d92bb-aa15-4446-aaef-ad8fdd3ce076 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:16.871598 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d27d92bb-aa15-4446-aaef-ad8fdd3ce076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:16.968198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:17.126292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533825c7-0d1c-4ded-b47e-cca5ad71d327 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:17.127799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-533825c7-0d1c-4ded-b47e-cca5ad71d327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:17.177486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:17.282969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533825c7-0d1c-4ded-b47e-cca5ad71d327 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:17.400600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70390c-a836-494d-9dc2-1837d379be77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:17.417602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70390c-a836-494d-9dc2-1837d379be77 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:17.417602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70390c-a836-494d-9dc2-1837d379be77 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:17.417602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70390c-a836-494d-9dc2-1837d379be77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:17.417602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bc70390c-a836-494d-9dc2-1837d379be77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:17.417602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:17.496918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70390c-a836-494d-9dc2-1837d379be77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:17.498586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70390c-a836-494d-9dc2-1837d379be77 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:17.517494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 709/1431] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:18.406326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a631fb8-a8b5-4b7e-b62f-0b5484465c54 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:18.407260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a631fb8-a8b5-4b7e-b62f-0b5484465c54 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:18.407260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a631fb8-a8b5-4b7e-b62f-0b5484465c54 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.408234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:18.413406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364dfe-db42-4019-b6d2-15c4658eb9b3 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:18.413804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364dfe-db42-4019-b6d2-15c4658eb9b3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:18.413939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364dfe-db42-4019-b6d2-15c4658eb9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:18.491381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-70364dfe-db42-4019-b6d2-15c4658eb9b3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:18.492574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-70364dfe-db42-4019-b6d2-15c4658eb9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:18.562035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:18.591237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:18.743811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a631fb8-a8b5-4b7e-b62f-0b5484465c54 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:18.745269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7a631fb8-a8b5-4b7e-b62f-0b5484465c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:18.763022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:18.893014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a631fb8-a8b5-4b7e-b62f-0b5484465c54 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:18.981910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:19.008150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd9e2b-42de-4687-bebb-4f09434313d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:19.008674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd9e2b-42de-4687-bebb-4f09434313d5 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.008943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd9e2b-42de-4687-bebb-4f09434313d5 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.009204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd9e2b-42de-4687-bebb-4f09434313d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:19.010788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4abd9e2b-42de-4687-bebb-4f09434313d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:19.103334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd9e2b-42de-4687-bebb-4f09434313d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:19.104931 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd9e2b-42de-4687-bebb-4f09434313d5 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:19.123109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 724/1434] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:19.998243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d7eb7-d0d9-4086-90ff-292bb45f2b80 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:19.998794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d7eb7-d0d9-4086-90ff-292bb45f2b80 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:19.998794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d7eb7-d0d9-4086-90ff-292bb45f2b80 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:20.000171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:20.006450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ddcb0-0732-45ba-a7c7-983ccadd4720 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:20.006888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ddcb0-0732-45ba-a7c7-983ccadd4720 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:20.007097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ddcb0-0732-45ba-a7c7-983ccadd4720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:20.083026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-775ddcb0-0732-45ba-a7c7-983ccadd4720 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:20.084232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-775ddcb0-0732-45ba-a7c7-983ccadd4720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:20.127174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:20.184470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:20.335420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d7eb7-d0d9-4086-90ff-292bb45f2b80 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:20.336939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a55d7eb7-d0d9-4086-90ff-292bb45f2b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:20.489990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d7eb7-d0d9-4086-90ff-292bb45f2b80 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:20.545236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:20.606319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6672e7b-bcbc-43b1-90a2-7bc288d6955c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:20.606842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6672e7b-bcbc-43b1-90a2-7bc288d6955c service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:20.607139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6672e7b-bcbc-43b1-90a2-7bc288d6955c service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:20.607406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6672e7b-bcbc-43b1-90a2-7bc288d6955c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:20.608996 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b6672e7b-bcbc-43b1-90a2-7bc288d6955c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:20.698152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6672e7b-bcbc-43b1-90a2-7bc288d6955c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:20.699774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6672e7b-bcbc-43b1-90a2-7bc288d6955c service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:20.717798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 712/1437] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:21.596260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edd9f5-bae6-4033-8982-3913b6510d4f None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:21.596756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edd9f5-bae6-4033-8982-3913b6510d4f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:21.596952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edd9f5-bae6-4033-8982-3913b6510d4f None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:21.598619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:21.603673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da8464-0ad5-413f-8662-81e73c722221 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:21.604109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da8464-0ad5-413f-8662-81e73c722221 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:21.604215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da8464-0ad5-413f-8662-81e73c722221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:21.682469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-24da8464-0ad5-413f-8662-81e73c722221 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:21.683707 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-24da8464-0ad5-413f-8662-81e73c722221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:21.685740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:21.786075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:21.936535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edd9f5-bae6-4033-8982-3913b6510d4f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:21.938270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-79edd9f5-bae6-4033-8982-3913b6510d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:22.089155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edd9f5-bae6-4033-8982-3913b6510d4f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:22.116997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:22.209030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315060f1-2603-4a77-a603-ddf40c64fa1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:22.209552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315060f1-2603-4a77-a603-ddf40c64fa1e service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:22.209820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315060f1-2603-4a77-a603-ddf40c64fa1e service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:22.210084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315060f1-2603-4a77-a603-ddf40c64fa1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:22.211687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-315060f1-2603-4a77-a603-ddf40c64fa1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:22.300882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315060f1-2603-4a77-a603-ddf40c64fa1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:22.302474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315060f1-2603-4a77-a603-ddf40c64fa1e service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:22.321264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 727/1440] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:24.166735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80809656-e6ac-4939-8d28-e599adef47bd None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.167307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80809656-e6ac-4939-8d28-e599adef47bd None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.167407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80809656-e6ac-4939-8d28-e599adef47bd None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.168670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:24.174065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0911cf-f205-4c2e-981b-1b0c99e406e3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.174498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0911cf-f205-4c2e-981b-1b0c99e406e3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.174617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0911cf-f205-4c2e-981b-1b0c99e406e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.217394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:24.243694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7f0911cf-f205-4c2e-981b-1b0c99e406e3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:24.244901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7f0911cf-f205-4c2e-981b-1b0c99e406e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.344938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:24.636728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80809656-e6ac-4939-8d28-e599adef47bd None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:24.638303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-80809656-e6ac-4939-8d28-e599adef47bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.742895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80809656-e6ac-4939-8d28-e599adef47bd None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:24.775524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:24.821258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b11cc8-1e85-4de2-b537-8e04f22e0440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:24.821898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11cc8-1e85-4de2-b537-8e04f22e0440 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:24.822173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11cc8-1e85-4de2-b537-8e04f22e0440 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:24.822440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11cc8-1e85-4de2-b537-8e04f22e0440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:24.824144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a9b11cc8-1e85-4de2-b537-8e04f22e0440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:24.939772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b11cc8-1e85-4de2-b537-8e04f22e0440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:24.941218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b11cc8-1e85-4de2-b537-8e04f22e0440 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:24.960119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 715/1443] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:27.044007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d3e1b-3668-46b7-a650-c99838b05296 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.044577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d3e1b-3668-46b7-a650-c99838b05296 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.044577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d3e1b-3668-46b7-a650-c99838b05296 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.046025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:27.051740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668084-cff0-4cd1-80c3-d854d0977fbf None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.052190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668084-cff0-4cd1-80c3-d854d0977fbf None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.052306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668084-cff0-4cd1-80c3-d854d0977fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.120068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5a668084-cff0-4cd1-80c3-d854d0977fbf None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:27.121263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5a668084-cff0-4cd1-80c3-d854d0977fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:27.134446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:27.223539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:27.513035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d3e1b-3668-46b7-a650-c99838b05296 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:27.514585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-720d3e1b-3668-46b7-a650-c99838b05296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:27.621553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d3e1b-3668-46b7-a650-c99838b05296 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:27.634259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:27.701296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2d7df-674c-4056-b588-abd585597784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:27.701899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2d7df-674c-4056-b588-abd585597784 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:27.702171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2d7df-674c-4056-b588-abd585597784 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:27.702434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2d7df-674c-4056-b588-abd585597784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:27.704078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-40e2d7df-674c-4056-b588-abd585597784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:27.815635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e2d7df-674c-4056-b588-abd585597784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:27.817775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e2d7df-674c-4056-b588-abd585597784 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:27.835666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 730/1446] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:30.139079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0d6c3-a56e-4555-a6ff-b0c0d2983d1f None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.139779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0d6c3-a56e-4555-a6ff-b0c0d2983d1f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.139779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0d6c3-a56e-4555-a6ff-b0c0d2983d1f None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.140970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:30.147270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5683f63-8cf4-4990-89fd-6d313c390b42 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.147711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5683f63-8cf4-4990-89fd-6d313c390b42 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.147814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5683f63-8cf4-4990-89fd-6d313c390b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.199282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:30.217012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e5683f63-8cf4-4990-89fd-6d313c390b42 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:30.218191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e5683f63-8cf4-4990-89fd-6d313c390b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:30.318829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:50:30.621161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0d6c3-a56e-4555-a6ff-b0c0d2983d1f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:30.622727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-46b0d6c3-a56e-4555-a6ff-b0c0d2983d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:30.728492 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0d6c3-a56e-4555-a6ff-b0c0d2983d1f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:30.769929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:30.804992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ae20e-ec05-49d6-ae44-d76ac4c7b0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:30.809894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ae20e-ec05-49d6-ae44-d76ac4c7b0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:30.809894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ae20e-ec05-49d6-ae44-d76ac4c7b0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:30.809894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ae20e-ec05-49d6-ae44-d76ac4c7b0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:30.809894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-451ae20e-ec05-49d6-ae44-d76ac4c7b0d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:30.923775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ae20e-ec05-49d6-ae44-d76ac4c7b0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:30.925910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ae20e-ec05-49d6-ae44-d76ac4c7b0d6 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:30.944858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 718/1449] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:33.028520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f63ae-8b28-4ded-a394-c612a7eef95e None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:33.029086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f63ae-8b28-4ded-a394-c612a7eef95e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:33.029086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f63ae-8b28-4ded-a394-c612a7eef95e None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:33.030395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:33.036865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd86f9-823c-4144-bb9e-53bc40b76004 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:33.037364 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd86f9-823c-4144-bb9e-53bc40b76004 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:33.037457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd86f9-823c-4144-bb9e-53bc40b76004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:33.104134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-80dd86f9-823c-4144-bb9e-53bc40b76004 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:33.105334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-80dd86f9-823c-4144-bb9e-53bc40b76004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:33.125709 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:33.203982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:33.493889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f63ae-8b28-4ded-a394-c612a7eef95e None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:33.495519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b49f63ae-8b28-4ded-a394-c612a7eef95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:33.603734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f63ae-8b28-4ded-a394-c612a7eef95e None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:33.624087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:33.678121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460452b9-d4da-47d2-9983-c5180be7d1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:33.678721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460452b9-d4da-47d2-9983-c5180be7d1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:33.679059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460452b9-d4da-47d2-9983-c5180be7d1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:33.679375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460452b9-d4da-47d2-9983-c5180be7d1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:33.680957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-460452b9-d4da-47d2-9983-c5180be7d1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:33.791869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460452b9-d4da-47d2-9983-c5180be7d1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:33.793356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460452b9-d4da-47d2-9983-c5180be7d1b0 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:33.817198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 733/1452] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:35.801967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeac138-57c3-45e8-88ca-0364d8964a5c None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.802528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeac138-57c3-45e8-88ca-0364d8964a5c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.802528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeac138-57c3-45e8-88ca-0364d8964a5c None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.803909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:35.809112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d9fd6-61e4-4a86-896a-f901645638c6 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:35.809561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d9fd6-61e4-4a86-896a-f901645638c6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:35.809666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d9fd6-61e4-4a86-896a-f901645638c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:35.875346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:35.878569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5d7d9fd6-61e4-4a86-896a-f901645638c6 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:35.879825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5d7d9fd6-61e4-4a86-896a-f901645638c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:35.977048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:36.269618 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeac138-57c3-45e8-88ca-0364d8964a5c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:36.271288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4aeac138-57c3-45e8-88ca-0364d8964a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:36.376730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeac138-57c3-45e8-88ca-0364d8964a5c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:36.427884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:36.453211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc86de-3f83-4b91-a371-dfb73813e3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:36.453814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc86de-3f83-4b91-a371-dfb73813e3ad service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:36.454090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc86de-3f83-4b91-a371-dfb73813e3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:36.454365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc86de-3f83-4b91-a371-dfb73813e3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:36.456802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-61cc86de-3f83-4b91-a371-dfb73813e3ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:36.567606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc86de-3f83-4b91-a371-dfb73813e3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:36.568953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc86de-3f83-4b91-a371-dfb73813e3ad service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:36.588393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 721/1455] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:38.549449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0915f32-269d-4523-93b0-cc41ba306b09 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.550170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0915f32-269d-4523-93b0-cc41ba306b09 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.550170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0915f32-269d-4523-93b0-cc41ba306b09 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.551866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:38.557222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f00694-0b32-452a-bee6-f664256f924f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:38.557673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f00694-0b32-452a-bee6-f664256f924f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:38.557797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f00694-0b32-452a-bee6-f664256f924f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:38.624993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-23f00694-0b32-452a-bee6-f664256f924f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:38.626209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-23f00694-0b32-452a-bee6-f664256f924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:38.656762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:38.726372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:39.016987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0915f32-269d-4523-93b0-cc41ba306b09 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:39.019096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b0915f32-269d-4523-93b0-cc41ba306b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:39.124739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0915f32-269d-4523-93b0-cc41ba306b09 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:39.154584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:39.202177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9866370-0399-4341-968e-a152bc47217f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:39.202755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9866370-0399-4341-968e-a152bc47217f service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:39.203060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9866370-0399-4341-968e-a152bc47217f service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:39.203337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9866370-0399-4341-968e-a152bc47217f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:39.204947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f9866370-0399-4341-968e-a152bc47217f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:39.316198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9866370-0399-4341-968e-a152bc47217f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:39.317573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9866370-0399-4341-968e-a152bc47217f service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:39.336963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 736/1458] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:41.338555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5e17e-ff62-41ef-b150-2f14d021027f None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:41.339249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5e17e-ff62-41ef-b150-2f14d021027f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:41.339249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5e17e-ff62-41ef-b150-2f14d021027f None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:41.340757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:41.345951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c6019-9c0d-4f0f-aa90-efdb1c2a46f7 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:41.346360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c6019-9c0d-4f0f-aa90-efdb1c2a46f7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:41.346490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c6019-9c0d-4f0f-aa90-efdb1c2a46f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:41.413404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cd4c6019-9c0d-4f0f-aa90-efdb1c2a46f7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:41.414599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cd4c6019-9c0d-4f0f-aa90-efdb1c2a46f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:41.422806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:41.512951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:41.796972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5e17e-ff62-41ef-b150-2f14d021027f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:41.798559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-51e5e17e-ff62-41ef-b150-2f14d021027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:41.905756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5e17e-ff62-41ef-b150-2f14d021027f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:41.962246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:41.982237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e242f36-9c20-4c53-b616-db1d334db3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:41.982848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242f36-9c20-4c53-b616-db1d334db3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:41.983154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242f36-9c20-4c53-b616-db1d334db3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:41.983429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242f36-9c20-4c53-b616-db1d334db3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:41.985115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2e242f36-9c20-4c53-b616-db1d334db3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:42.098209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e242f36-9c20-4c53-b616-db1d334db3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:42.099664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e242f36-9c20-4c53-b616-db1d334db3d1 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:42.118539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 724/1461] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:44.116378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53d4ea-8c36-4e12-b7e5-2cf5a70653e2 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:44.116997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53d4ea-8c36-4e12-b7e5-2cf5a70653e2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:44.116997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53d4ea-8c36-4e12-b7e5-2cf5a70653e2 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:44.118263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:44.123860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aa96a-df95-4369-aef5-156097c24130 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:44.124317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aa96a-df95-4369-aef5-156097c24130 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:44.124542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aa96a-df95-4369-aef5-156097c24130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:44.193102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8c5aa96a-df95-4369-aef5-156097c24130 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:44.194322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8c5aa96a-df95-4369-aef5-156097c24130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:44.234846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:44.293930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:44.582471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53d4ea-8c36-4e12-b7e5-2cf5a70653e2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:44.584035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ce53d4ea-8c36-4e12-b7e5-2cf5a70653e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:44.687805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53d4ea-8c36-4e12-b7e5-2cf5a70653e2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:44.726862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:44.764246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda5fe1-60d7-416c-b8e7-2676c7945c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:44.764847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5fe1-60d7-416c-b8e7-2676c7945c82 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:44.765120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5fe1-60d7-416c-b8e7-2676c7945c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:44.765387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda5fe1-60d7-416c-b8e7-2676c7945c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:44.767005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9bda5fe1-60d7-416c-b8e7-2676c7945c82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:44.878020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda5fe1-60d7-416c-b8e7-2676c7945c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:44.879494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda5fe1-60d7-416c-b8e7-2676c7945c82 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:44.897899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 739/1464] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:46.843248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9382-c88b-4f33-949b-a131d7803cb3 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:46.843811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9382-c88b-4f33-949b-a131d7803cb3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:46.843811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9382-c88b-4f33-949b-a131d7803cb3 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:46.845103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:50:46.851246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00931e-74e4-4077-82fa-ea5647c4af13 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:46.851695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00931e-74e4-4077-82fa-ea5647c4af13 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:46.851810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00931e-74e4-4077-82fa-ea5647c4af13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:46.917848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4f00931e-74e4-4077-82fa-ea5647c4af13 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:46.919131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4f00931e-74e4-4077-82fa-ea5647c4af13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:46.937875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:47.019422 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:47.307915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f9382-c88b-4f33-949b-a131d7803cb3 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:47.309488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-758f9382-c88b-4f33-949b-a131d7803cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:47.418117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f9382-c88b-4f33-949b-a131d7803cb3 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:47.487832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:47.495346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385eb6e8-47e3-4bb2-8d9f-0c55e5261ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:47.495984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eb6e8-47e3-4bb2-8d9f-0c55e5261ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:47.496259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eb6e8-47e3-4bb2-8d9f-0c55e5261ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:47.496529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385eb6e8-47e3-4bb2-8d9f-0c55e5261ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:47.498167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-385eb6e8-47e3-4bb2-8d9f-0c55e5261ad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:47.610467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385eb6e8-47e3-4bb2-8d9f-0c55e5261ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:47.611933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385eb6e8-47e3-4bb2-8d9f-0c55e5261ad7 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:47.629657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 727/1467] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:49.685755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98106fd6-8b7f-4793-828c-cfe164b36631 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.686287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98106fd6-8b7f-4793-828c-cfe164b36631 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.686287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98106fd6-8b7f-4793-828c-cfe164b36631 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.687583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:50:49.693947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59169c02-6b67-4f97-a376-90ac0d13433d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:49.694438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59169c02-6b67-4f97-a376-90ac0d13433d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:49.694546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59169c02-6b67-4f97-a376-90ac0d13433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:49.761222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-59169c02-6b67-4f97-a376-90ac0d13433d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:50:49.762439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-59169c02-6b67-4f97-a376-90ac0d13433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:49.811908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:49.861719 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:50:50.146524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98106fd6-8b7f-4793-828c-cfe164b36631 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:50:50.148117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-98106fd6-8b7f-4793-828c-cfe164b36631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:50.252899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98106fd6-8b7f-4793-828c-cfe164b36631 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:50:50.302449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:50:50.328443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36daf3c5-e42b-478b-841e-b37f795ee413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:50:50.329050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daf3c5-e42b-478b-841e-b37f795ee413 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:50:50.329320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daf3c5-e42b-478b-841e-b37f795ee413 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:50:50.329584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daf3c5-e42b-478b-841e-b37f795ee413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:50:50.332912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-36daf3c5-e42b-478b-841e-b37f795ee413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:50:50.442903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36daf3c5-e42b-478b-841e-b37f795ee413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:50:50.444285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36daf3c5-e42b-478b-841e-b37f795ee413 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:50:50.463093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 742/1470] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:50:59.859881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=22 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:00.460786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=66 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:10.396550 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1359b9-2c30-4f38-b949-487896015a60 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:10.397093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1359b9-2c30-4f38-b949-487896015a60 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:10.397093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1359b9-2c30-4f38-b949-487896015a60 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:10.398441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 729/1471] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:51:10.403927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418eab3-9499-4bf4-a8b5-9d257c0e9124 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:10.404729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418eab3-9499-4bf4-a8b5-9d257c0e9124 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:10.404729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418eab3-9499-4bf4-a8b5-9d257c0e9124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:10.471992 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2418eab3-9499-4bf4-a8b5-9d257c0e9124 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:10.473114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2418eab3-9499-4bf4-a8b5-9d257c0e9124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:10.574576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 743/1472] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:10.878004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2e936-a7ec-4054-b65d-6b6e88c7db90 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:10.878280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2e936-a7ec-4054-b65d-6b6e88c7db90 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:10.878548 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2e936-a7ec-4054-b65d-6b6e88c7db90 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:10.879928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 730/1473] 10.222.0.94 () {54 vars in 967 bytes} [Fri Aug 26 15:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:51:10.884765 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80d3c-4f1f-419f-8fb5-47f545778285 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:10.885081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80d3c-4f1f-419f-8fb5-47f545778285 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:10.885354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80d3c-4f1f-419f-8fb5-47f545778285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:10.963537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-96e80d3c-4f1f-419f-8fb5-47f545778285 None None] MFA Rules not processed for user `23ddf8f600454da7ae9b715cd4cbd186`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:10.965003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-96e80d3c-4f1f-419f-8fb5-47f545778285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:10.987759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:11.044565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 744/1474] 10.222.0.94 () {58 vars in 1064 bytes} [Fri Aug 26 15:51:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:11.053303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2e936-a7ec-4054-b65d-6b6e88c7db90 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:11.054857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bce2e936-a7ec-4054-b65d-6b6e88c7db90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:11.160583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2e936-a7ec-4054-b65d-6b6e88c7db90 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:11.234387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b7031-3c10-4a55-8823-c1cacd3bdc4e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23ddf8f600454da7ae9b715cd4cbd186', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:11.235020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b7031-3c10-4a55-8823-c1cacd3bdc4e service glance] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:11.235298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b7031-3c10-4a55-8823-c1cacd3bdc4e service glance] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:11.235567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b7031-3c10-4a55-8823-c1cacd3bdc4e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:11.237803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d79b7031-3c10-4a55-8823-c1cacd3bdc4e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:11.293539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:11.473412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b7031-3c10-4a55-8823-c1cacd3bdc4e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:11.473412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b7031-3c10-4a55-8823-c1cacd3bdc4e service glance] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:11.489859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 731/1475] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:51:13.536504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32e85-3627-4176-9c17-b3f91758cbb6 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.537109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32e85-3627-4176-9c17-b3f91758cbb6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.537109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32e85-3627-4176-9c17-b3f91758cbb6 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.538414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 745/1476] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:51:13.543896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c82b91-e46e-4783-886f-e72fc6519d0e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:13.544330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c82b91-e46e-4783-886f-e72fc6519d0e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:13.544455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c82b91-e46e-4783-886f-e72fc6519d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:13.611823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-98c82b91-e46e-4783-886f-e72fc6519d0e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:13.613044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-98c82b91-e46e-4783-886f-e72fc6519d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:13.708979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 732/1477] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:13.984964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32e85-3627-4176-9c17-b3f91758cbb6 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:13.987131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-99a32e85-3627-4176-9c17-b3f91758cbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:14.113905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32e85-3627-4176-9c17-b3f91758cbb6 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:14.163089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:14.207866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28c3b2-0bca-4429-9f49-849957ea5a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:14.208468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28c3b2-0bca-4429-9f49-849957ea5a6f admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:14.208742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28c3b2-0bca-4429-9f49-849957ea5a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:14.209021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28c3b2-0bca-4429-9f49-849957ea5a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:14.209702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28c3b2-0bca-4429-9f49-849957ea5a6f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:14.211476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING py.warnings [None req-ca28c3b2-0bca-4429-9f49-849957ea5a6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:51:14.211476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: warnings.warn(msg) Aug 26 15:51:14.211476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: Aug 26 15:51:14.212588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28c3b2-0bca-4429-9f49-849957ea5a6f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:14.232975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 746/1478] 10.222.0.94 () {56 vars in 1108 bytes} [Fri Aug 26 15:51:13 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:51:16.187588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc39a-4d82-482e-822f-a8358975f4b3 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.188157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc39a-4d82-482e-822f-a8358975f4b3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.188157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc39a-4d82-482e-822f-a8358975f4b3 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.189931 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 733/1479] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:51:16.195521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938bd8a-162b-417f-b36e-326ee5d2e2e1 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.196021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938bd8a-162b-417f-b36e-326ee5d2e2e1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.196129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938bd8a-162b-417f-b36e-326ee5d2e2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.269207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7938bd8a-162b-417f-b36e-326ee5d2e2e1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:16.270426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7938bd8a-162b-417f-b36e-326ee5d2e2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:16.371214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 747/1480] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:51:16.672522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dc39a-4d82-482e-822f-a8358975f4b3 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:16.674228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0b8dc39a-4d82-482e-822f-a8358975f4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:16.684027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:16.831052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dc39a-4d82-482e-822f-a8358975f4b3 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:16.916040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:16.949666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69c0b3-417a-4d86-9cd9-bd07b06f2e08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:16.950302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69c0b3-417a-4d86-9cd9-bd07b06f2e08 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:16.950579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69c0b3-417a-4d86-9cd9-bd07b06f2e08 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:16.950845 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69c0b3-417a-4d86-9cd9-bd07b06f2e08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:16.952524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bc69c0b3-417a-4d86-9cd9-bd07b06f2e08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:17.052041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69c0b3-417a-4d86-9cd9-bd07b06f2e08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:17.053616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69c0b3-417a-4d86-9cd9-bd07b06f2e08 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:17.072221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 734/1481] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:19.219367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac919608-f6d5-45dc-b9a8-713fe56ea01c None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:19.220435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac919608-f6d5-45dc-b9a8-713fe56ea01c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:19.220435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac919608-f6d5-45dc-b9a8-713fe56ea01c None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:19.222122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 748/1482] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:51:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:51:19.227570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae131ab-50fc-4c87-b9da-00716dd1b349 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:19.228040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae131ab-50fc-4c87-b9da-00716dd1b349 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:19.228151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae131ab-50fc-4c87-b9da-00716dd1b349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:19.296107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-bae131ab-50fc-4c87-b9da-00716dd1b349 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:19.297321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bae131ab-50fc-4c87-b9da-00716dd1b349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:19.351558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:19.403154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 735/1483] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:19.700584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac919608-f6d5-45dc-b9a8-713fe56ea01c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:19.702187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ac919608-f6d5-45dc-b9a8-713fe56ea01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:19.727522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:19.810280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac919608-f6d5-45dc-b9a8-713fe56ea01c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:19.885885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2090b-00f9-40b4-9710-44af5baf2e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:19.886490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2090b-00f9-40b4-9710-44af5baf2e82 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:19.886764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2090b-00f9-40b4-9710-44af5baf2e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:19.887056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2090b-00f9-40b4-9710-44af5baf2e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:19.888941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b8b2090b-00f9-40b4-9710-44af5baf2e82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:19.965473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:20.001680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2090b-00f9-40b4-9710-44af5baf2e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:20.003187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2090b-00f9-40b4-9710-44af5baf2e82 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:20.023025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 749/1484] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:21.994202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f1687-89c4-4f95-9950-6453ed710681 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:21.994772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f1687-89c4-4f95-9950-6453ed710681 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:21.994772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f1687-89c4-4f95-9950-6453ed710681 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:21.996137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 736/1485] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:51:22.002599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83c550-2b7c-4960-82a2-33014e0eb329 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.003078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83c550-2b7c-4960-82a2-33014e0eb329 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.003185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83c550-2b7c-4960-82a2-33014e0eb329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.074380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2e83c550-2b7c-4960-82a2-33014e0eb329 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:22.075621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2e83c550-2b7c-4960-82a2-33014e0eb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:22.176411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 750/1486] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:22.478480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f1687-89c4-4f95-9950-6453ed710681 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:22.480103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a05f1687-89c4-4f95-9950-6453ed710681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:22.588348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f1687-89c4-4f95-9950-6453ed710681 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:22.661561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:22.663442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae20e2a-1a1f-4be9-a4fb-05c823ffd1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:22.664055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae20e2a-1a1f-4be9-a4fb-05c823ffd1ff service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:22.664332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae20e2a-1a1f-4be9-a4fb-05c823ffd1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:22.664600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae20e2a-1a1f-4be9-a4fb-05c823ffd1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:22.666254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3ae20e2a-1a1f-4be9-a4fb-05c823ffd1ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:22.778270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae20e2a-1a1f-4be9-a4fb-05c823ffd1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:22.779651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae20e2a-1a1f-4be9-a4fb-05c823ffd1ff service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:22.799530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 737/1487] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:24.905882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec5656-ab3e-4ae3-9b48-6b14813613e3 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:24.906451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec5656-ab3e-4ae3-9b48-6b14813613e3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:24.906451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec5656-ab3e-4ae3-9b48-6b14813613e3 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:24.908475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 751/1488] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:51:24.913786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e798a-0d5c-4ac8-86ce-891fe884d882 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:24.914193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e798a-0d5c-4ac8-86ce-891fe884d882 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:24.914328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e798a-0d5c-4ac8-86ce-891fe884d882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:24.981306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-873e798a-0d5c-4ac8-86ce-891fe884d882 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:24.982521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-873e798a-0d5c-4ac8-86ce-891fe884d882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:25.035440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:25.080093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 738/1489] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:25.374565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec5656-ab3e-4ae3-9b48-6b14813613e3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:25.376750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-13ec5656-ab3e-4ae3-9b48-6b14813613e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:25.414255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:25.486905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec5656-ab3e-4ae3-9b48-6b14813613e3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:25.559991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a10134-cb0c-496c-9196-4b5c7ffeb730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:25.560612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10134-cb0c-496c-9196-4b5c7ffeb730 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:25.560890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10134-cb0c-496c-9196-4b5c7ffeb730 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:25.561162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10134-cb0c-496c-9196-4b5c7ffeb730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:25.562799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c9a10134-cb0c-496c-9196-4b5c7ffeb730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:25.647137 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:25.677381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a10134-cb0c-496c-9196-4b5c7ffeb730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:25.678913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a10134-cb0c-496c-9196-4b5c7ffeb730 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:25.698032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 752/1490] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:28.049403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18867fd8-cdb1-4827-a977-b73fbf90cfaa None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.050015 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18867fd8-cdb1-4827-a977-b73fbf90cfaa None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.050015 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18867fd8-cdb1-4827-a977-b73fbf90cfaa None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.051376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 739/1491] 10.222.0.94 () {54 vars in 927 bytes} [Fri Aug 26 15:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:51:28.056884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0c2b-dac3-4232-b703-c1e185f4cf99 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.057348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0c2b-dac3-4232-b703-c1e185f4cf99 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.057439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0c2b-dac3-4232-b703-c1e185f4cf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.124166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-12ac0c2b-dac3-4232-b703-c1e185f4cf99 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:51:28.125376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-12ac0c2b-dac3-4232-b703-c1e185f4cf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.224383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 753/1492] 10.222.0.94 () {58 vars in 1024 bytes} [Fri Aug 26 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:51:28.525266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18867fd8-cdb1-4827-a977-b73fbf90cfaa None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:51:28.526834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-18867fd8-cdb1-4827-a977-b73fbf90cfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.633252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18867fd8-cdb1-4827-a977-b73fbf90cfaa None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:51:28.707345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bede73-9c2b-4200-a892-8577e2d9f60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:51:28.707948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bede73-9c2b-4200-a892-8577e2d9f60e service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:51:28.708222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bede73-9c2b-4200-a892-8577e2d9f60e service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:51:28.708489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bede73-9c2b-4200-a892-8577e2d9f60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:51:28.710094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-29bede73-9c2b-4200-a892-8577e2d9f60e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:51:28.719520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:28.820220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bede73-9c2b-4200-a892-8577e2d9f60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:51:28.821569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bede73-9c2b-4200-a892-8577e2d9f60e service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:51:28.839698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 740/1493] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:51:38.222561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=82 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:51:38.837140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=50 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:52.948718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d721d6-02d9-4641-84f2-be2a1b96fb3f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:52.949314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d721d6-02d9-4641-84f2-be2a1b96fb3f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:52.949314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d721d6-02d9-4641-84f2-be2a1b96fb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:53.026978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e5d721d6-02d9-4641-84f2-be2a1b96fb3f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:53.028188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e5d721d6-02d9-4641-84f2-be2a1b96fb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:53.125376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 754/1494] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:53.134356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bac0a-045b-423d-88d2-a2a61ac5eb50 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:53.135078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bac0a-045b-423d-88d2-a2a61ac5eb50 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:53.135078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bac0a-045b-423d-88d2-a2a61ac5eb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:53.213392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ab6bac0a-045b-423d-88d2-a2a61ac5eb50 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:53.214544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ab6bac0a-045b-423d-88d2-a2a61ac5eb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:53.312407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 741/1495] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:53.317730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d721d6-02d9-4641-84f2-be2a1b96fb3f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:53.318992 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e5d721d6-02d9-4641-84f2-be2a1b96fb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:53.330700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d721d6-02d9-4641-84f2-be2a1b96fb3f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:53.398256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:53.434641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:53.435353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:53.435636 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:53.435901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:53.436557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:52:53.436854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:53.437878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING py.warnings [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:52:53.437878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: warnings.warn(msg) Aug 26 15:52:53.437878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: Aug 26 15:52:53.439599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:53.451902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 755/1496] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:52:53.461642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bac0a-045b-423d-88d2-a2a61ac5eb50 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:53.473749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bac0a-045b-423d-88d2-a2a61ac5eb50 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:53.548588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:53.583160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:53.583768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:53.584035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:53.584296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:53.584906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:53.586897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WARNING py.warnings [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:52:53.586897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: warnings.warn(msg) Aug 26 15:52:53.586897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: Aug 26 15:52:53.588149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:53.713596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 742/1497] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:52:53 2022] POST /identity/v3/projects => generated 379 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:53.717944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:53.720090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:53.754545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef37c1-4bad-485b-acdd-63d9ee9c44b8 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:53.849130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:53.849741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:53.850019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:53.850289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:53.850914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:53.851959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING py.warnings [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:52:53.851959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: warnings.warn(msg) Aug 26 15:52:53.851959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: Aug 26 15:52:53.853163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:53.980472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 756/1498] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:52:53 2022] POST /identity/v3/users => generated 290 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:52:53.984703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:53.986856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:54.051913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff61bf-57af-4e19-a69c-17f6e06a7638 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:54.135235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:54.145870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:54.146474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:54.146751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:54.147050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:54.147689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:54.148703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WARNING py.warnings [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:52:54.148703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: warnings.warn(msg) Aug 26 15:52:54.148703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: Aug 26 15:52:54.149587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:54.162480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 743/1499] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:52:54.166991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:54.168582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:54.180749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2329417-72c5-4bb9-be84-0a75b77057aa admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:54.206759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:54.279459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb368-581c-46ea-85f7-581538545ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:54.280063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb368-581c-46ea-85f7-581538545ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:54.280333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb368-581c-46ea-85f7-581538545ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:54.280595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb368-581c-46ea-85f7-581538545ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:54.281226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb368-581c-46ea-85f7-581538545ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:54.282234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING py.warnings [None req-68cdb368-581c-46ea-85f7-581538545ac4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:52:54.282234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: warnings.warn(msg) Aug 26 15:52:54.282234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: Aug 26 15:52:54.283718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb368-581c-46ea-85f7-581538545ac4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:54.296160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 757/1500] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:52:54.300946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:54.302558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:54.338519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d28cd5-d9e8-479d-9422-da45e94e152d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:54.434515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce1db8-6b3a-4669-aec5-c1c145853387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:54.435198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1db8-6b3a-4669-aec5-c1c145853387 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:54.435480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1db8-6b3a-4669-aec5-c1c145853387 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:54.435743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce1db8-6b3a-4669-aec5-c1c145853387 admin admin] PATH_INFO: `/v3/projects/1822c5d1b4324e3c93d0489d66ed3ae4/users/3c623a6620d74601920f036da857e68a/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:54.459341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce1db8-6b3a-4669-aec5-c1c145853387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1822c5d1b4324e3c93d0489d66ed3ae4, user_id=3c623a6620d74601920f036da857e68a, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:54.460540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WARNING py.warnings [None req-91ce1db8-6b3a-4669-aec5-c1c145853387 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:52:54.460540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: warnings.warn(msg) Aug 26 15:52:54.460540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: Aug 26 15:52:54.462733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce1db8-6b3a-4669-aec5-c1c145853387 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:54.557647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 744/1501] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:52:54 2022] PUT /identity/v3/projects/1822c5d1b4324e3c93d0489d66ed3ae4/users/3c623a6620d74601920f036da857e68a/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:52:54.570427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965effa3-402f-4400-a89e-3241cd7a9000 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:54.570712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965effa3-402f-4400-a89e-3241cd7a9000 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:54.571016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965effa3-402f-4400-a89e-3241cd7a9000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:54.629538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-965effa3-402f-4400-a89e-3241cd7a9000 None None] MFA Rules not processed for user `3c623a6620d74601920f036da857e68a`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:54.630764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-965effa3-402f-4400-a89e-3241cd7a9000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:54.708722 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 758/1502] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:52:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:54.722864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdb9a8-95db-453b-80d2-6cb69784397e None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:54.723172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdb9a8-95db-453b-80d2-6cb69784397e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:54.723444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdb9a8-95db-453b-80d2-6cb69784397e None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:54.724780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 745/1503] 10.222.0.94 () {54 vars in 964 bytes} [Fri Aug 26 15:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:52:54.729557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084280ad-d9f1-44dc-bbfe-50fb8d4bf59c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:54.729843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084280ad-d9f1-44dc-bbfe-50fb8d4bf59c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:54.730149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084280ad-d9f1-44dc-bbfe-50fb8d4bf59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:54.762782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:54.808204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-084280ad-d9f1-44dc-bbfe-50fb8d4bf59c None None] MFA Rules not processed for user `58567e28a5f741eda021aecab43d6a63`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:54.809349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-084280ad-d9f1-44dc-bbfe-50fb8d4bf59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:54.886209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 759/1504] 10.222.0.94 () {58 vars in 1061 bytes} [Fri Aug 26 15:52:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:54.894693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdb9a8-95db-453b-80d2-6cb69784397e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:54.896261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-90bdb9a8-95db-453b-80d2-6cb69784397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:54.946671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:55.001958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdb9a8-95db-453b-80d2-6cb69784397e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:55.074960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:55.075490 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:55.075764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:55.076030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:55.077617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:55.182072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:55.183577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:55.187197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:55.202309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 746/1505] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:55.248926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689240a-0d67-4b27-bd18-1f6ebeec864e None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:55.249376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689240a-0d67-4b27-bd18-1f6ebeec864e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:55.249499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689240a-0d67-4b27-bd18-1f6ebeec864e None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:55.251300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 760/1506] 10.222.0.94 () {52 vars in 830 bytes} [Fri Aug 26 15:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:52:55.255015 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:55.256247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:55.361289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c418c2-fb65-478f-8b80-fc0252aec0eb service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:55.459472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:55.460105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:55.460382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:55.460654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] PATH_INFO: `/v3/users/3c623a6620d74601920f036da857e68a` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:55.473836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c623a6620d74601920f036da857e68a)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:55.474871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WARNING py.warnings [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:52:55.474871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: warnings.warn(msg) Aug 26 15:52:55.474871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: Aug 26 15:52:55.476111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:55.486392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:55.669215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c623a6620d74601920f036da857e68a'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:55.669695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c623a6620d74601920f036da857e68a'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:55.693267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c623a6620d74601920f036da857e68a'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:55.693821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c623a6620d74601920f036da857e68a'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:55.771759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c623a6620d74601920f036da857e68a'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:55.792995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 747/1507] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:52:55 2022] DELETE /identity/v3/users/3c623a6620d74601920f036da857e68a => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:52:55.797700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a086c-95df-40dd-b75b-b504c7d02775 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:55.797984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a086c-95df-40dd-b75b-b504c7d02775 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:55.798268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a086c-95df-40dd-b75b-b504c7d02775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:55.855544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a97a086c-95df-40dd-b75b-b504c7d02775 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:52:55.856847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a97a086c-95df-40dd-b75b-b504c7d02775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:55.932762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:55.945198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 761/1508] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:52:55.956130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:55.957349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:56.058565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7701df-bbbe-41d4-8214-d2dce1293799 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:56.115122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:52:56.134629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763ed06-84b3-4a8d-8865-770f17817430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:56.135274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ed06-84b3-4a8d-8865-770f17817430 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:56.135553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ed06-84b3-4a8d-8865-770f17817430 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:56.135825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763ed06-84b3-4a8d-8865-770f17817430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:56.137457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1763ed06-84b3-4a8d-8865-770f17817430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:56.252900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763ed06-84b3-4a8d-8865-770f17817430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:56.254259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763ed06-84b3-4a8d-8865-770f17817430 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:56.272802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 748/1509] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:52:56.764905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a086c-95df-40dd-b75b-b504c7d02775 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:52:56.766179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a97a086c-95df-40dd-b75b-b504c7d02775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:52:56.778710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a086c-95df-40dd-b75b-b504c7d02775 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:52:56.880325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:52:56.880965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:52:56.881241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:52:56.881509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] PATH_INFO: `/v3/projects/1822c5d1b4324e3c93d0489d66ed3ae4` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:52:56.892096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1822c5d1b4324e3c93d0489d66ed3ae4)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:52:56.893136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING py.warnings [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:52:56.893136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: warnings.warn(msg) Aug 26 15:52:56.893136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: Aug 26 15:52:56.894343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:52:57.036484 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1822c5d1b4324e3c93d0489d66ed3ae4', 'request_id': 'req-ce287c51-4eed-4714-8d55-b76161ea208f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:57.042460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1822c5d1b4324e3c93d0489d66ed3ae4', 'request_id': 'req-ce287c51-4eed-4714-8d55-b76161ea208f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:57.112064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1822c5d1b4324e3c93d0489d66ed3ae4', 'request_id': 'req-ce287c51-4eed-4714-8d55-b76161ea208f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:57.144945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] The token cache is being invalidate because project 1822c5d1b4324e3c93d0489d66ed3ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:52:57.145221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:57.145751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ce287c51-4eed-4714-8d55-b76161ea208f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:52:57.147114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 762/1510] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:52:56 2022] DELETE /identity/v3/projects/1822c5d1b4324e3c93d0489d66ed3ae4 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:53:06.270522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=62 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:53:07.142863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:54:34.089383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970d72d-473f-43af-8707-3e3913571d2e None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:34.090248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970d72d-473f-43af-8707-3e3913571d2e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:34.090248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970d72d-473f-43af-8707-3e3913571d2e None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:34.091314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 15:54:34.096369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf704e71-33a1-4e9e-8974-00d7c9e1a35a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:34.096831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf704e71-33a1-4e9e-8974-00d7c9e1a35a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:34.096949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf704e71-33a1-4e9e-8974-00d7c9e1a35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:34.179110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cf704e71-33a1-4e9e-8974-00d7c9e1a35a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:34.180317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cf704e71-33a1-4e9e-8974-00d7c9e1a35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:34.286623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:54:34.448023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970d72d-473f-43af-8707-3e3913571d2e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:34.450252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5970d72d-473f-43af-8707-3e3913571d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:34.572543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970d72d-473f-43af-8707-3e3913571d2e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:34.578123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff40c11-660d-4fb6-950d-1b484f008127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:34.578754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40c11-660d-4fb6-950d-1b484f008127 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:34.579059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40c11-660d-4fb6-950d-1b484f008127 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:34.579327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40c11-660d-4fb6-950d-1b484f008127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:34.588674 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff40c11-660d-4fb6-950d-1b484f008127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:34.590252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff40c11-660d-4fb6-950d-1b484f008127 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:34.600187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 750/1513] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:54:35.131961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cae673-3c65-4c51-bc7c-0c72ed752aa6 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:35.132533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cae673-3c65-4c51-bc7c-0c72ed752aa6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:35.132533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cae673-3c65-4c51-bc7c-0c72ed752aa6 None None] PATH_INFO: `/v3` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:35.133809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 764/1514] 10.222.0.102 () {54 vars in 926 bytes} [Fri Aug 26 15:54:35 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:54:35.140359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc9375-d78c-431f-a606-05897bca015a None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:35.140790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc9375-d78c-431f-a606-05897bca015a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:35.141032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc9375-d78c-431f-a606-05897bca015a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:35.202987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-39cc9375-d78c-431f-a606-05897bca015a None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:35.204200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-39cc9375-d78c-431f-a606-05897bca015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:35.275801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:54:35.281423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 751/1515] 10.222.0.102 () {58 vars in 1016 bytes} [Fri Aug 26 15:54:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:35.458703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e6a2-8b7e-4ba7-9dfa-a39eff6490e4 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:35.458956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e6a2-8b7e-4ba7-9dfa-a39eff6490e4 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:35.459188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e6a2-8b7e-4ba7-9dfa-a39eff6490e4 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:35.460537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 765/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:54:35.465533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cec15-7b8d-46ce-b11a-7a0103ebadc5 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:35.465820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cec15-7b8d-46ce-b11a-7a0103ebadc5 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:35.466097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cec15-7b8d-46ce-b11a-7a0103ebadc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:35.485842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-011cec15-7b8d-46ce-b11a-7a0103ebadc5 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:35.487435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-011cec15-7b8d-46ce-b11a-7a0103ebadc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:35.496496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 752/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:35.653074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608e6a2-8b7e-4ba7-9dfa-a39eff6490e4 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:35.668901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608e6a2-8b7e-4ba7-9dfa-a39eff6490e4 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:35.674216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff8fe5-278a-441a-8c7c-322cf6b01f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:35.674802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff8fe5-278a-441a-8c7c-322cf6b01f2d service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:35.675106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff8fe5-278a-441a-8c7c-322cf6b01f2d service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:35.675377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff8fe5-278a-441a-8c7c-322cf6b01f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:35.683750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff8fe5-278a-441a-8c7c-322cf6b01f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:35.685334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff8fe5-278a-441a-8c7c-322cf6b01f2d service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:35.695416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 766/1518] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:54:44.636693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63c0fe-7178-4275-83ec-6df1e5446d9e None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:44.637218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63c0fe-7178-4275-83ec-6df1e5446d9e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:44.637218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63c0fe-7178-4275-83ec-6df1e5446d9e None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:44.638527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 753/1519] 10.222.0.94 () {54 vars in 961 bytes} [Fri Aug 26 15:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:54:44.643385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4338a-a4d2-4795-be59-384af78150b6 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:44.643826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4338a-a4d2-4795-be59-384af78150b6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:44.643959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4338a-a4d2-4795-be59-384af78150b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:44.664790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6cc4338a-a4d2-4795-be59-384af78150b6 None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:44.665985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6cc4338a-a4d2-4795-be59-384af78150b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:44.674526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 767/1520] 10.222.0.94 () {58 vars in 1058 bytes} [Fri Aug 26 15:54:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:44.682769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63c0fe-7178-4275-83ec-6df1e5446d9e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:44.696038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63c0fe-7178-4275-83ec-6df1e5446d9e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:44.700993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95301dbe-c0a0-4384-9fdc-6d4d68c3e64f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:44.701572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301dbe-c0a0-4384-9fdc-6d4d68c3e64f service placement] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:44.701844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301dbe-c0a0-4384-9fdc-6d4d68c3e64f service placement] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:44.702111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301dbe-c0a0-4384-9fdc-6d4d68c3e64f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:44.710767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95301dbe-c0a0-4384-9fdc-6d4d68c3e64f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:44.712363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95301dbe-c0a0-4384-9fdc-6d4d68c3e64f service placement] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:44.721888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 754/1521] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:54:46.916582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10665bbb-f6d1-48ee-b789-1cb0f757a5b1 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:46.917164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10665bbb-f6d1-48ee-b789-1cb0f757a5b1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:46.917164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10665bbb-f6d1-48ee-b789-1cb0f757a5b1 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:46.918479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:54:46.923384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3f05-d986-4f5e-bf38-1bfed8c888d7 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:46.923812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3f05-d986-4f5e-bf38-1bfed8c888d7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:46.923918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3f05-d986-4f5e-bf38-1bfed8c888d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:46.944182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d62e3f05-d986-4f5e-bf38-1bfed8c888d7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:46.945380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d62e3f05-d986-4f5e-bf38-1bfed8c888d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:46.953229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:47.110372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10665bbb-f6d1-48ee-b789-1cb0f757a5b1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:47.125674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10665bbb-f6d1-48ee-b789-1cb0f757a5b1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:47.131805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffc0cb-b3d9-4416-9762-85463f2cbb01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:47.132518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffc0cb-b3d9-4416-9762-85463f2cbb01 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:47.132793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffc0cb-b3d9-4416-9762-85463f2cbb01 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:47.133064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffc0cb-b3d9-4416-9762-85463f2cbb01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:47.141395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffc0cb-b3d9-4416-9762-85463f2cbb01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:47.143034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffc0cb-b3d9-4416-9762-85463f2cbb01 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:47.153104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 769/1524] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:54:47.997392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094de17-4db4-4b3d-8ac1-2a50298495eb None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:47.997976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094de17-4db4-4b3d-8ac1-2a50298495eb None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:47.997976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094de17-4db4-4b3d-8ac1-2a50298495eb None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:47.999349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 15:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:54:48.004762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8fc68-8e4b-453a-971c-3988f83950c2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:48.005156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8fc68-8e4b-453a-971c-3988f83950c2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:48.005315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8fc68-8e4b-453a-971c-3988f83950c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:48.024354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-82f8fc68-8e4b-453a-971c-3988f83950c2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:48.025542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-82f8fc68-8e4b-453a-971c-3988f83950c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:48.033444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 15:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:48.186973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094de17-4db4-4b3d-8ac1-2a50298495eb None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:48.201884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094de17-4db4-4b3d-8ac1-2a50298495eb None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:48.208259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28cc79-113f-431f-abd8-ce5c4fafc390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:48.208853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cc79-113f-431f-abd8-ce5c4fafc390 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:48.209129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cc79-113f-431f-abd8-ce5c4fafc390 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:48.209399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cc79-113f-431f-abd8-ce5c4fafc390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:48.217649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28cc79-113f-431f-abd8-ce5c4fafc390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:48.219262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28cc79-113f-431f-abd8-ce5c4fafc390 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:48.228797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 757/1527] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:54:52.418697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bca1cd-11f6-45bb-839b-35ea57039af9 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:52.419340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bca1cd-11f6-45bb-839b-35ea57039af9 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:52.419340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bca1cd-11f6-45bb-839b-35ea57039af9 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:52.420667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:54:52.426147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513a3d9-68f3-4b36-8a90-3fee151347ae None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:52.426558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513a3d9-68f3-4b36-8a90-3fee151347ae None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:52.426694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513a3d9-68f3-4b36-8a90-3fee151347ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:52.446207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f513a3d9-68f3-4b36-8a90-3fee151347ae None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:52.447437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f513a3d9-68f3-4b36-8a90-3fee151347ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:52.455360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:52.748358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bca1cd-11f6-45bb-839b-35ea57039af9 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:52.750037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-39bca1cd-11f6-45bb-839b-35ea57039af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:52.784178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:54:52.832702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bca1cd-11f6-45bb-839b-35ea57039af9 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:52.836617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60cbf0-1d95-4aa2-b1e0-63277659aa17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:52.837223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60cbf0-1d95-4aa2-b1e0-63277659aa17 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:52.837498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60cbf0-1d95-4aa2-b1e0-63277659aa17 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:52.837768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60cbf0-1d95-4aa2-b1e0-63277659aa17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:52.846652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60cbf0-1d95-4aa2-b1e0-63277659aa17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:52.848317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60cbf0-1d95-4aa2-b1e0-63277659aa17 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:52.858031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 772/1530] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:54:54.795918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980faa80-1b0a-4c29-bda0-c26169897179 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:54.796483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980faa80-1b0a-4c29-bda0-c26169897179 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:54.796483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980faa80-1b0a-4c29-bda0-c26169897179 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:54.797820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:54:54.803730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6a8b-9121-4526-a265-6b0084025112 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:54.804178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6a8b-9121-4526-a265-6b0084025112 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:54.804290 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6a8b-9121-4526-a265-6b0084025112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:54.824331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d4ce6a8b-9121-4526-a265-6b0084025112 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:54.825581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d4ce6a8b-9121-4526-a265-6b0084025112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:54.833635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:55.121528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980faa80-1b0a-4c29-bda0-c26169897179 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:55.133625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980faa80-1b0a-4c29-bda0-c26169897179 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:55.137440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440316fc-b159-481b-8691-3a2caeb7a701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:55.138042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440316fc-b159-481b-8691-3a2caeb7a701 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:55.138318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440316fc-b159-481b-8691-3a2caeb7a701 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:55.138584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440316fc-b159-481b-8691-3a2caeb7a701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:55.149711 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440316fc-b159-481b-8691-3a2caeb7a701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:55.151224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440316fc-b159-481b-8691-3a2caeb7a701 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:55.168335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 760/1533] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:54:58.874895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce270-ea0e-46b9-9f56-a6251ac6065b None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:58.875462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce270-ea0e-46b9-9f56-a6251ac6065b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:58.875462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce270-ea0e-46b9-9f56-a6251ac6065b None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:58.877030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:54:58.883115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3500d-2a00-4c5f-9912-cfabadbb279a None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:58.883576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3500d-2a00-4c5f-9912-cfabadbb279a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:58.883686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3500d-2a00-4c5f-9912-cfabadbb279a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:58.903883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0da3500d-2a00-4c5f-9912-cfabadbb279a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:54:58.908318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0da3500d-2a00-4c5f-9912-cfabadbb279a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:58.913350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:54:59.467897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efce270-ea0e-46b9-9f56-a6251ac6065b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:54:59.469575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2efce270-ea0e-46b9-9f56-a6251ac6065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:54:59.550638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efce270-ea0e-46b9-9f56-a6251ac6065b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:54:59.554235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82b73f-ee27-4549-aa8c-0d2243d5bdce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:54:59.554992 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82b73f-ee27-4549-aa8c-0d2243d5bdce service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:54:59.555274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82b73f-ee27-4549-aa8c-0d2243d5bdce service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:54:59.555546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82b73f-ee27-4549-aa8c-0d2243d5bdce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:54:59.565003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82b73f-ee27-4549-aa8c-0d2243d5bdce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:54:59.569549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82b73f-ee27-4549-aa8c-0d2243d5bdce service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:54:59.576975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 775/1536] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:54:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:01.535238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54165c-b6b3-46bd-a98d-1ff2c1b3ce17 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:01.535753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54165c-b6b3-46bd-a98d-1ff2c1b3ce17 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:01.535753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54165c-b6b3-46bd-a98d-1ff2c1b3ce17 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:01.537117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 15:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:55:01.542345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252fede-e046-4d8a-9c0a-38b67f1deb26 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:01.542750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252fede-e046-4d8a-9c0a-38b67f1deb26 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:01.542888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252fede-e046-4d8a-9c0a-38b67f1deb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:01.562640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5252fede-e046-4d8a-9c0a-38b67f1deb26 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:55:01.563858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5252fede-e046-4d8a-9c0a-38b67f1deb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:01.572410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 15:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:55:02.126567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54165c-b6b3-46bd-a98d-1ff2c1b3ce17 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:55:02.129048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ac54165c-b6b3-46bd-a98d-1ff2c1b3ce17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:55:02.142566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54165c-b6b3-46bd-a98d-1ff2c1b3ce17 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:55:02.146738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4d99a-aa52-4c66-92e2-fcb96db59328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:55:02.147358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4d99a-aa52-4c66-92e2-fcb96db59328 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:55:02.147635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4d99a-aa52-4c66-92e2-fcb96db59328 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:55:02.148051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4d99a-aa52-4c66-92e2-fcb96db59328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:55:02.157117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4d99a-aa52-4c66-92e2-fcb96db59328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:55:02.163444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4d99a-aa52-4c66-92e2-fcb96db59328 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:55:02.180778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 763/1539] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:55:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:55:11.557884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=63 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:55:12.172583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=37 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:29.442824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384dc38-8b97-4a60-a78d-dfe5e4ec611b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.443481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384dc38-8b97-4a60-a78d-dfe5e4ec611b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.443481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384dc38-8b97-4a60-a78d-dfe5e4ec611b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.473580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-9384dc38-8b97-4a60-a78d-dfe5e4ec611b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.474879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9384dc38-8b97-4a60-a78d-dfe5e4ec611b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.487646 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 777/1540] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.496849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8932e-4add-439a-abf3-68280ed021dc None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.497907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8932e-4add-439a-abf3-68280ed021dc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.498420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8932e-4add-439a-abf3-68280ed021dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.528532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2278c-3c0d-419b-a579-d147f11fe461 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.528809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2278c-3c0d-419b-a579-d147f11fe461 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.529254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2278c-3c0d-419b-a579-d147f11fe461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.534834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-65e8932e-4add-439a-abf3-68280ed021dc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.536421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-65e8932e-4add-439a-abf3-68280ed021dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.549284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 764/1541] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.553860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8932e-4add-439a-abf3-68280ed021dc None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:29.574946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-55b2278c-3c0d-419b-a579-d147f11fe461 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.574946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-55b2278c-3c0d-419b-a579-d147f11fe461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.586441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8932e-4add-439a-abf3-68280ed021dc None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:29.596067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79132e2d-ea90-4b6a-91d8-a31ab7710682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:29.597061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79132e2d-ea90-4b6a-91d8-a31ab7710682 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.597568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79132e2d-ea90-4b6a-91d8-a31ab7710682 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.598069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79132e2d-ea90-4b6a-91d8-a31ab7710682 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.598968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79132e2d-ea90-4b6a-91d8-a31ab7710682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:29.599500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79132e2d-ea90-4b6a-91d8-a31ab7710682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:29.601111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WARNING py.warnings [None req-79132e2d-ea90-4b6a-91d8-a31ab7710682 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:56:29.601111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: warnings.warn(msg) Aug 26 15:56:29.601111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: Aug 26 15:56:29.602403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 778/1542] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:56:29.602717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79132e2d-ea90-4b6a-91d8-a31ab7710682 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:29.612870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76951a09-4668-4bc6-858a-a0bfa96ab60d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.613130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76951a09-4668-4bc6-858a-a0bfa96ab60d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.613411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76951a09-4668-4bc6-858a-a0bfa96ab60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.628056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 765/1543] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:56:29.629764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647231-d06f-4129-8250-8dcd158dc153 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.630291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647231-d06f-4129-8250-8dcd158dc153 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.630798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647231-d06f-4129-8250-8dcd158dc153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.656914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-83647231-d06f-4129-8250-8dcd158dc153 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.658712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-83647231-d06f-4129-8250-8dcd158dc153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.666885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-76951a09-4668-4bc6-858a-a0bfa96ab60d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.666885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-76951a09-4668-4bc6-858a-a0bfa96ab60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.675758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 766/1544] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.677029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c2e90-effe-4aff-aa63-a64d171907b8 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.677307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c2e90-effe-4aff-aa63-a64d171907b8 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.677579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c2e90-effe-4aff-aa63-a64d171907b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.686798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 779/1545] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.688964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76951a09-4668-4bc6-858a-a0bfa96ab60d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:29.702153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76951a09-4668-4bc6-858a-a0bfa96ab60d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:29.705795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7d8c2e90-effe-4aff-aa63-a64d171907b8 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.707444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7d8c2e90-effe-4aff-aa63-a64d171907b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.709515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343ae5a-1d51-466c-a738-3c79dbcdb96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:29.710214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343ae5a-1d51-466c-a738-3c79dbcdb96c admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.710657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343ae5a-1d51-466c-a738-3c79dbcdb96c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.711039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343ae5a-1d51-466c-a738-3c79dbcdb96c admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.711675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343ae5a-1d51-466c-a738-3c79dbcdb96c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:29.712706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING py.warnings [None req-3343ae5a-1d51-466c-a738-3c79dbcdb96c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:56:29.712706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: warnings.warn(msg) Aug 26 15:56:29.712706 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: Aug 26 15:56:29.713899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343ae5a-1d51-466c-a738-3c79dbcdb96c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:29.715952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 767/1546] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.717179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e034-9bba-41aa-946b-afaa4a875427 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.717454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e034-9bba-41aa-946b-afaa4a875427 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.717720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2e034-9bba-41aa-946b-afaa4a875427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.737573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e7f2e034-9bba-41aa-946b-afaa4a875427 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.738759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e7f2e034-9bba-41aa-946b-afaa4a875427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.746596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 768/1547] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.747957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715028-265d-4c73-bff2-e771e8b2aa59 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.748235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715028-265d-4c73-bff2-e771e8b2aa59 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.748496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715028-265d-4c73-bff2-e771e8b2aa59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.768677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-06715028-265d-4c73-bff2-e771e8b2aa59 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.769853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-06715028-265d-4c73-bff2-e771e8b2aa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.777603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 769/1548] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.778895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c791db-c332-4f04-9f3a-234335bf48fc None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.779207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c791db-c332-4f04-9f3a-234335bf48fc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.779472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c791db-c332-4f04-9f3a-234335bf48fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.798832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-b6c791db-c332-4f04-9f3a-234335bf48fc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.800000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b6c791db-c332-4f04-9f3a-234335bf48fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.808385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 770/1549] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.809205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c791db-c332-4f04-9f3a-234335bf48fc None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:29.823455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c791db-c332-4f04-9f3a-234335bf48fc None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:29.831184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ee325-89b8-48b6-b556-a7f45558f066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:29.831890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee325-89b8-48b6-b556-a7f45558f066 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.832168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee325-89b8-48b6-b556-a7f45558f066 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.832436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ee325-89b8-48b6-b556-a7f45558f066 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.833079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee325-89b8-48b6-b556-a7f45558f066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:29.833373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee325-89b8-48b6-b556-a7f45558f066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:29.835072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ee325-89b8-48b6-b556-a7f45558f066 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:29.839903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 780/1550] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:29.846423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d290859-85a5-4f84-9644-c154951fa27c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.846423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d290859-85a5-4f84-9644-c154951fa27c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.846423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d290859-85a5-4f84-9644-c154951fa27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.848534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 771/1551] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:56:29.849940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df087acf-0f05-42c4-9151-18b9305853ec None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.850222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df087acf-0f05-42c4-9151-18b9305853ec None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.850493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df087acf-0f05-42c4-9151-18b9305853ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.863173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2d290859-85a5-4f84-9644-c154951fa27c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.864389 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2d290859-85a5-4f84-9644-c154951fa27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.870195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-df087acf-0f05-42c4-9151-18b9305853ec None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.871397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-df087acf-0f05-42c4-9151-18b9305853ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.925732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 781/1552] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.927721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a476-7ca0-4369-b9bc-72528e9e959a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.928238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a476-7ca0-4369-b9bc-72528e9e959a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.928742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5a476-7ca0-4369-b9bc-72528e9e959a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.934176 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 772/1553] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.935097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df087acf-0f05-42c4-9151-18b9305853ec None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:29.945902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df087acf-0f05-42c4-9151-18b9305853ec None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:29.950912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295d097-c8e7-4612-9100-7d9dac3fa90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:29.951643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d097-c8e7-4612-9100-7d9dac3fa90c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.951873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d097-c8e7-4612-9100-7d9dac3fa90c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.952147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d097-c8e7-4612-9100-7d9dac3fa90c admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.953297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295d097-c8e7-4612-9100-7d9dac3fa90c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:29.953599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295d097-c8e7-4612-9100-7d9dac3fa90c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:29.954444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-03d5a476-7ca0-4369-b9bc-72528e9e959a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.955683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-03d5a476-7ca0-4369-b9bc-72528e9e959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.956098 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295d097-c8e7-4612-9100-7d9dac3fa90c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:29.963934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 782/1554] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:29.965113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a17a6-0c6c-4b32-8eab-8638c150f823 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.965432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a17a6-0c6c-4b32-8eab-8638c150f823 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.965711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a17a6-0c6c-4b32-8eab-8638c150f823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.975059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 773/1555] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:56:29.976286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621009-7f1d-4b9d-ae51-66d5f653e297 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:29.976564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621009-7f1d-4b9d-ae51-66d5f653e297 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:29.976829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621009-7f1d-4b9d-ae51-66d5f653e297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:29.993608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-295a17a6-0c6c-4b32-8eab-8638c150f823 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.994829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-295a17a6-0c6c-4b32-8eab-8638c150f823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:29.998142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0d621009-7f1d-4b9d-ae51-66d5f653e297 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:29.999445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0d621009-7f1d-4b9d-ae51-66d5f653e297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:30.004193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 783/1556] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:30.004968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a17a6-0c6c-4b32-8eab-8638c150f823 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.011226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 774/1557] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:30.011226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d621009-7f1d-4b9d-ae51-66d5f653e297 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.017915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a17a6-0c6c-4b32-8eab-8638c150f823 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.021059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d621009-7f1d-4b9d-ae51-66d5f653e297 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.022804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51326-ba38-4167-bc31-b50d7227948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.023449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51326-ba38-4167-bc31-b50d7227948b admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.023819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51326-ba38-4167-bc31-b50d7227948b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.024146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51326-ba38-4167-bc31-b50d7227948b admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.024784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b51326-ba38-4167-bc31-b50d7227948b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.026196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200027d-9796-478c-a0ac-da331f8cb449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.026796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b51326-ba38-4167-bc31-b50d7227948b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.043162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200027d-9796-478c-a0ac-da331f8cb449 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.043162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200027d-9796-478c-a0ac-da331f8cb449 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.043162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200027d-9796-478c-a0ac-da331f8cb449 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.043162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200027d-9796-478c-a0ac-da331f8cb449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.043162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200027d-9796-478c-a0ac-da331f8cb449 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.117268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 784/1558] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.117488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 775/1559] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.117603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86a61d-c820-454c-8511-98361274ab82 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.117603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86a61d-c820-454c-8511-98361274ab82 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.117603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86a61d-c820-454c-8511-98361274ab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.117899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200027d-9796-478c-a0ac-da331f8cb449 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.137799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3e86a61d-c820-454c-8511-98361274ab82 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:30.139028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3e86a61d-c820-454c-8511-98361274ab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:30.173012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:30.185604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200027d-9796-478c-a0ac-da331f8cb449 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.191076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.191678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.191951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.192216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.192855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:30.193149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.194762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.201971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 785/1560] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:30.202898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86a61d-c820-454c-8511-98361274ab82 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.207308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 776/1561] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:56:30.208639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.213042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86a61d-c820-454c-8511-98361274ab82 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.217797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.220751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.220751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.220751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.221076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432f904-c6af-4ce7-9c24-1432f2b2f585 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.221186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:30.221186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.222460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.225198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491d42b-39f1-4924-99f1-39e5062f5276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.225791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491d42b-39f1-4924-99f1-39e5062f5276 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.226115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491d42b-39f1-4924-99f1-39e5062f5276 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.226444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491d42b-39f1-4924-99f1-39e5062f5276 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.227097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491d42b-39f1-4924-99f1-39e5062f5276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.228992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491d42b-39f1-4924-99f1-39e5062f5276 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.234985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 786/1562] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:56:30.236035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.246993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a6ec1-5ba8-4180-b1b6-534cf1b1dc20 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.252003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a95056-9347-40b8-9fb2-b6baf1b5a0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.252595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a95056-9347-40b8-9fb2-b6baf1b5a0cd admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.252868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a95056-9347-40b8-9fb2-b6baf1b5a0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.253132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a95056-9347-40b8-9fb2-b6baf1b5a0cd admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.253765 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a95056-9347-40b8-9fb2-b6baf1b5a0cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:30.254067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a95056-9347-40b8-9fb2-b6baf1b5a0cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.255689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a95056-9347-40b8-9fb2-b6baf1b5a0cd admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.257236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:30.268440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 787/1563] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:56:30.269728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010209fb-d8af-4086-a8a6-d1f3515b1c10 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.270007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010209fb-d8af-4086-a8a6-d1f3515b1c10 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.270274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010209fb-d8af-4086-a8a6-d1f3515b1c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.290146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-010209fb-d8af-4086-a8a6-d1f3515b1c10 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:30.291364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-010209fb-d8af-4086-a8a6-d1f3515b1c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:30.300361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 788/1564] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:30.301255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010209fb-d8af-4086-a8a6-d1f3515b1c10 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.309214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 777/1565] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.313989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491d42b-39f1-4924-99f1-39e5062f5276 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.314098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010209fb-d8af-4086-a8a6-d1f3515b1c10 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.375715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31eff-0b2e-47d9-a885-507d2a6dd90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.376278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31eff-0b2e-47d9-a885-507d2a6dd90c admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.376394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491d42b-39f1-4924-99f1-39e5062f5276 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.376555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31eff-0b2e-47d9-a885-507d2a6dd90c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.376821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e31eff-0b2e-47d9-a885-507d2a6dd90c admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.377463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e31eff-0b2e-47d9-a885-507d2a6dd90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.379091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e31eff-0b2e-47d9-a885-507d2a6dd90c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.381059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.381669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.381945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.382219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.382839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.383913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WARNING py.warnings [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:56:30.383913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: warnings.warn(msg) Aug 26 15:56:30.383913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: Aug 26 15:56:30.385116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.388975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 789/1566] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:30.389943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31eff-0b2e-47d9-a885-507d2a6dd90c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.400209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e31eff-0b2e-47d9-a885-507d2a6dd90c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.404748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.405327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.405601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.406696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.407672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:30.407973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.409562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.428871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 790/1567] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:56:30.428871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.432220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01436b7e-258b-434a-a7d8-ffb4a3cd00d1 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.436789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef8e82-8592-4fc0-a906-219555673bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.446026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef8e82-8592-4fc0-a906-219555673bbb admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.446026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef8e82-8592-4fc0-a906-219555673bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.446026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef8e82-8592-4fc0-a906-219555673bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.446026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef8e82-8592-4fc0-a906-219555673bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.446026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef8e82-8592-4fc0-a906-219555673bbb admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.530317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 778/1568] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.531357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.540960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021a49a-dcb6-41d4-b0cd-85c8cb8d86f9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.545657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043c2c-4d84-45df-8571-f9c2ff244c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.546225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043c2c-4d84-45df-8571-f9c2ff244c6c admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.547200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043c2c-4d84-45df-8571-f9c2ff244c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.547533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043c2c-4d84-45df-8571-f9c2ff244c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.548080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1043c2c-4d84-45df-8571-f9c2ff244c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.549978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1043c2c-4d84-45df-8571-f9c2ff244c6c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.567754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 791/1569] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.568597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef8e82-8592-4fc0-a906-219555673bbb admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.631680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef8e82-8592-4fc0-a906-219555673bbb admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.636185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6bc3b-28df-479d-9b09-3636093e53e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.636783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6bc3b-28df-479d-9b09-3636093e53e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.637060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6bc3b-28df-479d-9b09-3636093e53e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.637364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6bc3b-28df-479d-9b09-3636093e53e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.637982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6bc3b-28df-479d-9b09-3636093e53e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.639934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6bc3b-28df-479d-9b09-3636093e53e2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.646353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 779/1570] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.647297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043c2c-4d84-45df-8571-f9c2ff244c6c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.708750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043c2c-4d84-45df-8571-f9c2ff244c6c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.713313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.713938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.714212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.714478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.715141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:56:30.715439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.717019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.735629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 780/1571] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:56:30.735629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.740910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ebd4-b595-47d0-b0e7-c74a73005c97 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.745561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea30c9-1c89-4cf5-b76a-5744341535e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.746174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea30c9-1c89-4cf5-b76a-5744341535e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.746448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea30c9-1c89-4cf5-b76a-5744341535e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.746715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea30c9-1c89-4cf5-b76a-5744341535e1 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.747361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea30c9-1c89-4cf5-b76a-5744341535e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.749268 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea30c9-1c89-4cf5-b76a-5744341535e1 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.766450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 792/1572] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.773386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6bc3b-28df-479d-9b09-3636093e53e2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.801007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:30.829443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6bc3b-28df-479d-9b09-3636093e53e2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.834234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc088af-aaa9-4671-83ad-59edb412b94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.834839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc088af-aaa9-4671-83ad-59edb412b94d admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.835143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc088af-aaa9-4671-83ad-59edb412b94d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.835419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc088af-aaa9-4671-83ad-59edb412b94d admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.836103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc088af-aaa9-4671-83ad-59edb412b94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.837738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc088af-aaa9-4671-83ad-59edb412b94d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.839636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 781/1573] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.840299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea30c9-1c89-4cf5-b76a-5744341535e1 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.851248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 793/1574] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:30.851611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea30c9-1c89-4cf5-b76a-5744341535e1 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.852414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc088af-aaa9-4671-83ad-59edb412b94d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.856404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5dba4-954e-43f4-8dfa-3d18ca66deff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.856998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5dba4-954e-43f4-8dfa-3d18ca66deff admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.857278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5dba4-954e-43f4-8dfa-3d18ca66deff admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.857547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5dba4-954e-43f4-8dfa-3d18ca66deff admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.858167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5dba4-954e-43f4-8dfa-3d18ca66deff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.860373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5dba4-954e-43f4-8dfa-3d18ca66deff admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.863934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc088af-aaa9-4671-83ad-59edb412b94d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.865937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:30.870194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e3860-add1-4b37-aad6-e936df42c210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.870880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3860-add1-4b37-aad6-e936df42c210 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.871201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3860-add1-4b37-aad6-e936df42c210 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.871474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3860-add1-4b37-aad6-e936df42c210 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.872112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e3860-add1-4b37-aad6-e936df42c210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.873897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e3860-add1-4b37-aad6-e936df42c210 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.884235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 794/1575] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:30.884953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e3860-add1-4b37-aad6-e936df42c210 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:30.895041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e3860-add1-4b37-aad6-e936df42c210 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:30.900095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cc740-cb38-4884-8210-cb43904b6ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:30.900745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cc740-cb38-4884-8210-cb43904b6ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:30.901075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cc740-cb38-4884-8210-cb43904b6ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:30.901387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cc740-cb38-4884-8210-cb43904b6ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:30.902036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cc740-cb38-4884-8210-cb43904b6ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:30.904202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cc740-cb38-4884-8210-cb43904b6ce7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:30.972078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 782/1576] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:30.972914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5dba4-954e-43f4-8dfa-3d18ca66deff admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.018025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 795/1577] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:31.022043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cc740-cb38-4884-8210-cb43904b6ce7 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.031726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cc740-cb38-4884-8210-cb43904b6ce7 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.036415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d691e7d-d71d-4f6f-8ad4-74245dc0c77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.036999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d691e7d-d71d-4f6f-8ad4-74245dc0c77b admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.037271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d691e7d-d71d-4f6f-8ad4-74245dc0c77b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.037536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d691e7d-d71d-4f6f-8ad4-74245dc0c77b admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.038168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d691e7d-d71d-4f6f-8ad4-74245dc0c77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.038884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5dba4-954e-43f4-8dfa-3d18ca66deff admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.040085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d691e7d-d71d-4f6f-8ad4-74245dc0c77b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.043871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ac08b-1c78-44f2-a144-9ecc891c6706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.044478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ac08b-1c78-44f2-a144-9ecc891c6706 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.044755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ac08b-1c78-44f2-a144-9ecc891c6706 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.045021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ac08b-1c78-44f2-a144-9ecc891c6706 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.045637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ac08b-1c78-44f2-a144-9ecc891c6706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.048040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ac08b-1c78-44f2-a144-9ecc891c6706 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.113451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 783/1578] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:30 2022] POST /identity/v3/users => generated 283 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:31.114341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ac08b-1c78-44f2-a144-9ecc891c6706 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.114912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 796/1579] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:31 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:31.115587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d691e7d-d71d-4f6f-8ad4-74245dc0c77b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.128588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ac08b-1c78-44f2-a144-9ecc891c6706 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.181326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d691e7d-d71d-4f6f-8ad4-74245dc0c77b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.185938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545b454-ddac-49fa-ab9d-e6eba853eeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.186536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b454-ddac-49fa-ab9d-e6eba853eeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.186659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4205961-3de8-43de-af48-6c8cb9c9415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.186810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b454-ddac-49fa-ab9d-e6eba853eeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.187610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b454-ddac-49fa-ab9d-e6eba853eeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.187940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4205961-3de8-43de-af48-6c8cb9c9415d admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.188231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4205961-3de8-43de-af48-6c8cb9c9415d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.188346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545b454-ddac-49fa-ab9d-e6eba853eeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.188494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4205961-3de8-43de-af48-6c8cb9c9415d admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.189112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4205961-3de8-43de-af48-6c8cb9c9415d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.189873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545b454-ddac-49fa-ab9d-e6eba853eeb7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.191177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4205961-3de8-43de-af48-6c8cb9c9415d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.203781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 797/1580] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.203781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545b454-ddac-49fa-ab9d-e6eba853eeb7 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.211390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545b454-ddac-49fa-ab9d-e6eba853eeb7 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.215900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.216527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.216797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.217060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] PATH_INFO: `/v3/projects/8c69c9fb12124d72954b8ab7ad024175/users/7069a9b54a3245b3ab9671730344e6d3/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.231602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c69c9fb12124d72954b8ab7ad024175, user_id=7069a9b54a3245b3ab9671730344e6d3, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.232672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING py.warnings [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:56:31.232672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: warnings.warn(msg) Aug 26 15:56:31.232672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: Aug 26 15:56:31.234906 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.302196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 784/1581] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:31 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:31.303057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4205961-3de8-43de-af48-6c8cb9c9415d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.314126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4205961-3de8-43de-af48-6c8cb9c9415d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.318906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebd948-4544-4933-8749-5b984a11e587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.319511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebd948-4544-4933-8749-5b984a11e587 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.319790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebd948-4544-4933-8749-5b984a11e587 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.320063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebd948-4544-4933-8749-5b984a11e587 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.320699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebd948-4544-4933-8749-5b984a11e587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.322321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebd948-4544-4933-8749-5b984a11e587 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.333336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 785/1582] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.334312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebd948-4544-4933-8749-5b984a11e587 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.339712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 798/1583] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:31 2022] PUT /identity/v3/projects/8c69c9fb12124d72954b8ab7ad024175/users/7069a9b54a3245b3ab9671730344e6d3/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:31.340549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.344597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebd948-4544-4933-8749-5b984a11e587 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.396537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:31.396742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:31.402969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aea22-3ce0-4680-ac28-440282f6a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.403576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aea22-3ce0-4680-ac28-440282f6a32c admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.403856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aea22-3ce0-4680-ac28-440282f6a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.404147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aea22-3ce0-4680-ac28-440282f6a32c admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.404769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aea22-3ce0-4680-ac28-440282f6a32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.406766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aea22-3ce0-4680-ac28-440282f6a32c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.408192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf8fe0-3d2e-42b6-be7b-c7ab336b3c9a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.412833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cce30-4686-4bc0-8abb-c96c7c88822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.413464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cce30-4686-4bc0-8abb-c96c7c88822c admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.413734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cce30-4686-4bc0-8abb-c96c7c88822c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.414025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cce30-4686-4bc0-8abb-c96c7c88822c admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.414705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cce30-4686-4bc0-8abb-c96c7c88822c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.416314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cce30-4686-4bc0-8abb-c96c7c88822c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.427563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 799/1584] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.428501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cce30-4686-4bc0-8abb-c96c7c88822c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.439207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cce30-4686-4bc0-8abb-c96c7c88822c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.443794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baa67b-55eb-40a4-97e2-b22da61bbc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.444373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baa67b-55eb-40a4-97e2-b22da61bbc9f admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.444651 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baa67b-55eb-40a4-97e2-b22da61bbc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.444919 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6baa67b-55eb-40a4-97e2-b22da61bbc9f admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.445543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baa67b-55eb-40a4-97e2-b22da61bbc9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.447141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6baa67b-55eb-40a4-97e2-b22da61bbc9f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.457622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 800/1585] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.458385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baa67b-55eb-40a4-97e2-b22da61bbc9f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.467636 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6baa67b-55eb-40a4-97e2-b22da61bbc9f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.472350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3c77e-81b8-4623-a7fe-767b9b0f48b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.472909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3c77e-81b8-4623-a7fe-767b9b0f48b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.473188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3c77e-81b8-4623-a7fe-767b9b0f48b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.473460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3c77e-81b8-4623-a7fe-767b9b0f48b3 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.474058 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3c77e-81b8-4623-a7fe-767b9b0f48b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.475960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3c77e-81b8-4623-a7fe-767b9b0f48b3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.519426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 786/1586] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:31 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:31.520254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aea22-3ce0-4680-ac28-440282f6a32c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.529886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aea22-3ce0-4680-ac28-440282f6a32c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.534592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe9469-c941-4474-98ed-6a7775516160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.535209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe9469-c941-4474-98ed-6a7775516160 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.535471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe9469-c941-4474-98ed-6a7775516160 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.535742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe9469-c941-4474-98ed-6a7775516160 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.536375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe9469-c941-4474-98ed-6a7775516160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.537963 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe9469-c941-4474-98ed-6a7775516160 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.549585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 787/1587] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.550478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe9469-c941-4474-98ed-6a7775516160 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.560438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe9469-c941-4474-98ed-6a7775516160 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.565863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e853038-78af-42e2-9696-f41840decbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.566489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e853038-78af-42e2-9696-f41840decbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.566767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e853038-78af-42e2-9696-f41840decbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.567071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e853038-78af-42e2-9696-f41840decbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.567839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e853038-78af-42e2-9696-f41840decbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.569542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e853038-78af-42e2-9696-f41840decbc9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.580495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 788/1588] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.581377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e853038-78af-42e2-9696-f41840decbc9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.583536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 801/1589] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:31 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:31.589538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ae65-5127-4d56-9a7b-60e7eb184488 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.589538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ae65-5127-4d56-9a7b-60e7eb184488 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.589538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ae65-5127-4d56-9a7b-60e7eb184488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.591707 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e853038-78af-42e2-9696-f41840decbc9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.596748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921e1b-0464-42b7-a7be-1efe12564cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.597440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921e1b-0464-42b7-a7be-1efe12564cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.597720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921e1b-0464-42b7-a7be-1efe12564cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.597990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43921e1b-0464-42b7-a7be-1efe12564cc2 admin admin] PATH_INFO: `/v3/projects/5f170813ecff47a68d7610de9d769287/users/4940f8a9c15a483eab251df392789534/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.604079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2e71ae65-5127-4d56-9a7b-60e7eb184488 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:31.605287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2e71ae65-5127-4d56-9a7b-60e7eb184488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:31.613946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 802/1590] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:31.614661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43921e1b-0464-42b7-a7be-1efe12564cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f170813ecff47a68d7610de9d769287, user_id=4940f8a9c15a483eab251df392789534, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.614880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71ae65-5127-4d56-9a7b-60e7eb184488 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.617543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43921e1b-0464-42b7-a7be-1efe12564cc2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.625497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71ae65-5127-4d56-9a7b-60e7eb184488 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.630334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b55a95-3b1b-4581-9e75-b6e157bce1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.630954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55a95-3b1b-4581-9e75-b6e157bce1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.631289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55a95-3b1b-4581-9e75-b6e157bce1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.631579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55a95-3b1b-4581-9e75-b6e157bce1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.632219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b55a95-3b1b-4581-9e75-b6e157bce1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.633819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b55a95-3b1b-4581-9e75-b6e157bce1e6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.643919 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 803/1591] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.644827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b55a95-3b1b-4581-9e75-b6e157bce1e6 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.654461 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b55a95-3b1b-4581-9e75-b6e157bce1e6 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.659692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8d4c9-005b-4bbd-902f-8ad9ec986bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.660251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8d4c9-005b-4bbd-902f-8ad9ec986bfe admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.660528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8d4c9-005b-4bbd-902f-8ad9ec986bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.660798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8d4c9-005b-4bbd-902f-8ad9ec986bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.661409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8d4c9-005b-4bbd-902f-8ad9ec986bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.662968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8d4c9-005b-4bbd-902f-8ad9ec986bfe admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.673338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 804/1592] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.674217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8d4c9-005b-4bbd-902f-8ad9ec986bfe admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.683562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8d4c9-005b-4bbd-902f-8ad9ec986bfe admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.688861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a09bd-88cd-44eb-9739-367607fb45ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.689468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a09bd-88cd-44eb-9739-367607fb45ce admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.689697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a09bd-88cd-44eb-9739-367607fb45ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.689969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a09bd-88cd-44eb-9739-367607fb45ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.690577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a09bd-88cd-44eb-9739-367607fb45ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.692211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a09bd-88cd-44eb-9739-367607fb45ce admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.702519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 805/1593] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.703821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a09bd-88cd-44eb-9739-367607fb45ce admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.713585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a09bd-88cd-44eb-9739-367607fb45ce admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.713763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 789/1594] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:31 2022] PUT /identity/v3/projects/5f170813ecff47a68d7610de9d769287/users/4940f8a9c15a483eab251df392789534/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:31.714475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921e1b-0464-42b7-a7be-1efe12564cc2 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.773376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6a609-3b7e-42b7-b7c0-9b4e02c932ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.774033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a609-3b7e-42b7-b7c0-9b4e02c932ce admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.774307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a609-3b7e-42b7-b7c0-9b4e02c932ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.774568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a609-3b7e-42b7-b7c0-9b4e02c932ce admin admin] PATH_INFO: `/v3/projects/d95f731096d74a089f4161f5db69280c/users/b50041d36b0b430badd6442a2f09f063/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.777963 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43921e1b-0464-42b7-a7be-1efe12564cc2 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.783536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd18f76-c220-4d0d-b122-224e20be4675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.784142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd18f76-c220-4d0d-b122-224e20be4675 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.784421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd18f76-c220-4d0d-b122-224e20be4675 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.784691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd18f76-c220-4d0d-b122-224e20be4675 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.785338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd18f76-c220-4d0d-b122-224e20be4675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.786944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd18f76-c220-4d0d-b122-224e20be4675 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.790220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6a609-3b7e-42b7-b7c0-9b4e02c932ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95f731096d74a089f4161f5db69280c, user_id=b50041d36b0b430badd6442a2f09f063, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.792906 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6a609-3b7e-42b7-b7c0-9b4e02c932ce admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.796907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 790/1595] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.797732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd18f76-c220-4d0d-b122-224e20be4675 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.807235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd18f76-c220-4d0d-b122-224e20be4675 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.811835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9035af-30ef-46ea-87a8-274124e20fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.813318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9035af-30ef-46ea-87a8-274124e20fdf admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.813614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9035af-30ef-46ea-87a8-274124e20fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.813868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9035af-30ef-46ea-87a8-274124e20fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.814555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9035af-30ef-46ea-87a8-274124e20fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.816097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9035af-30ef-46ea-87a8-274124e20fdf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.825724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 791/1596] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:31.826494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9035af-30ef-46ea-87a8-274124e20fdf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.868255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 806/1597] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:31 2022] PUT /identity/v3/projects/d95f731096d74a089f4161f5db69280c/users/b50041d36b0b430badd6442a2f09f063/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:31.869059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6a609-3b7e-42b7-b7c0-9b4e02c932ce admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.886111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9035af-30ef-46ea-87a8-274124e20fdf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.933019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6a609-3b7e-42b7-b7c0-9b4e02c932ce admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.936079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:31.939812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842996d-ec91-4561-88ff-68b899b32864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.940414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842996d-ec91-4561-88ff-68b899b32864 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.940690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842996d-ec91-4561-88ff-68b899b32864 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.940958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842996d-ec91-4561-88ff-68b899b32864 admin admin] PATH_INFO: `/v3/projects/0d8be83798d04491b2e420af8502106f/users/80f71a262e95480582c37cd9541b41ba/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.944221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:31.945021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181964f-4ebc-4cdb-8c66-760eb09faa0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.945657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181964f-4ebc-4cdb-8c66-760eb09faa0d service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.947816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181964f-4ebc-4cdb-8c66-760eb09faa0d service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.947816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181964f-4ebc-4cdb-8c66-760eb09faa0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.956485 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181964f-4ebc-4cdb-8c66-760eb09faa0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.956655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842996d-ec91-4561-88ff-68b899b32864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8be83798d04491b2e420af8502106f, user_id=80f71a262e95480582c37cd9541b41ba, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:31.957929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181964f-4ebc-4cdb-8c66-760eb09faa0d service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.959282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842996d-ec91-4561-88ff-68b899b32864 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:31.967469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 792/1598] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:31.968838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181964f-4ebc-4cdb-8c66-760eb09faa0d service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:31.978285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181964f-4ebc-4cdb-8c66-760eb09faa0d service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:31.983147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30ae32-fe3a-4298-a005-9972e1d36a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:31.983791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ae32-fe3a-4298-a005-9972e1d36a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:31.984657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ae32-fe3a-4298-a005-9972e1d36a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:31.984929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ae32-fe3a-4298-a005-9972e1d36a48 admin admin] PATH_INFO: `/v3/projects/d6f7c7361aad41a594a0164ebfac2a0c/users/bf6426addc5f4cb49456968c16e2f370/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:31.999946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30ae32-fe3a-4298-a005-9972e1d36a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f7c7361aad41a594a0164ebfac2a0c, user_id=bf6426addc5f4cb49456968c16e2f370, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.003465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30ae32-fe3a-4298-a005-9972e1d36a48 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.063204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 807/1599] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:31 2022] PUT /identity/v3/projects/0d8be83798d04491b2e420af8502106f/users/80f71a262e95480582c37cd9541b41ba/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:32.064145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842996d-ec91-4561-88ff-68b899b32864 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.125762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842996d-ec91-4561-88ff-68b899b32864 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.132857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 793/1600] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:31 2022] PUT /identity/v3/projects/d6f7c7361aad41a594a0164ebfac2a0c/users/bf6426addc5f4cb49456968c16e2f370/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:32.134537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fc0f0-91b5-4e41-beda-9d12eb4e3a43 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.134825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fc0f0-91b5-4e41-beda-9d12eb4e3a43 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.135226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fc0f0-91b5-4e41-beda-9d12eb4e3a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.156913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-a33fc0f0-91b5-4e41-beda-9d12eb4e3a43 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:32.157970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a33fc0f0-91b5-4e41-beda-9d12eb4e3a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:32.198227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8c65b-62eb-4bfe-848a-b3aa9027b649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.198912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8c65b-62eb-4bfe-848a-b3aa9027b649 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.204732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8c65b-62eb-4bfe-848a-b3aa9027b649 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.204732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8c65b-62eb-4bfe-848a-b3aa9027b649 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.204732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8c65b-62eb-4bfe-848a-b3aa9027b649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.204732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8c65b-62eb-4bfe-848a-b3aa9027b649 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.217913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 808/1601] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:32.217913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8c65b-62eb-4bfe-848a-b3aa9027b649 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.229091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8c65b-62eb-4bfe-848a-b3aa9027b649 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.231576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 794/1602] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:32.235692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fc0f0-91b5-4e41-beda-9d12eb4e3a43 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.235817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654ae11-4888-44c6-9d6a-935e7da8067f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.235817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654ae11-4888-44c6-9d6a-935e7da8067f admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.235817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654ae11-4888-44c6-9d6a-935e7da8067f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.235817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654ae11-4888-44c6-9d6a-935e7da8067f admin admin] PATH_INFO: `/v3/projects/e8bfb4199b0d4a57933fe1b685aa01cd/users/228837054cd648acb0df93dd423e1d83/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.243928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fc0f0-91b5-4e41-beda-9d12eb4e3a43 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.248993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95de62-18ac-42f0-b30d-b2631a6b2f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.249638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95de62-18ac-42f0-b30d-b2631a6b2f97 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.250099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95de62-18ac-42f0-b30d-b2631a6b2f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.250548 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95de62-18ac-42f0-b30d-b2631a6b2f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.251095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654ae11-4888-44c6-9d6a-935e7da8067f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bfb4199b0d4a57933fe1b685aa01cd, user_id=228837054cd648acb0df93dd423e1d83, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.251233 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95de62-18ac-42f0-b30d-b2631a6b2f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.252834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95de62-18ac-42f0-b30d-b2631a6b2f97 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.253967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654ae11-4888-44c6-9d6a-935e7da8067f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.262899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 795/1603] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:32.264174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9ee56-05fd-4c20-84d0-24e2cbf76afa None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.264462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9ee56-05fd-4c20-84d0-24e2cbf76afa None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.264737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9ee56-05fd-4c20-84d0-24e2cbf76afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.284269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ace9ee56-05fd-4c20-84d0-24e2cbf76afa None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:32.285555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ace9ee56-05fd-4c20-84d0-24e2cbf76afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:32.293201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 796/1604] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:32.294537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873044e-d6f8-4b78-8ef0-fdb006a2b29f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.294819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873044e-d6f8-4b78-8ef0-fdb006a2b29f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.295115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873044e-d6f8-4b78-8ef0-fdb006a2b29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.319820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 809/1605] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:32 2022] PUT /identity/v3/projects/e8bfb4199b0d4a57933fe1b685aa01cd/users/228837054cd648acb0df93dd423e1d83/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:32.325154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed32b6-0d35-4865-8a5d-4c5846fbb4da None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.325154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed32b6-0d35-4865-8a5d-4c5846fbb4da None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.325154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed32b6-0d35-4865-8a5d-4c5846fbb4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.327453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-2873044e-d6f8-4b78-8ef0-fdb006a2b29f None None] MFA Rules not processed for user `80f71a262e95480582c37cd9541b41ba`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:32.329981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2873044e-d6f8-4b78-8ef0-fdb006a2b29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:32.354696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-30ed32b6-0d35-4865-8a5d-4c5846fbb4da None None] MFA Rules not processed for user `bf6426addc5f4cb49456968c16e2f370`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:32.355965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-30ed32b6-0d35-4865-8a5d-4c5846fbb4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:32.415140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 797/1606] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:32.415971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873044e-d6f8-4b78-8ef0-fdb006a2b29f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.435242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:32.443835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 810/1607] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:32.444965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed32b6-0d35-4865-8a5d-4c5846fbb4da None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.460577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:32.490481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873044e-d6f8-4b78-8ef0-fdb006a2b29f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.495196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf81399-61cb-45bb-8207-851ea842dab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.495858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf81399-61cb-45bb-8207-851ea842dab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.496131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf81399-61cb-45bb-8207-851ea842dab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.496406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf81399-61cb-45bb-8207-851ea842dab2 admin admin] PATH_INFO: `/v3/projects/5acbcd6961414a07a2fc96da3da3ef0a/users/17960d1d84f248fea5646a12bd6185e9/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.508331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed32b6-0d35-4865-8a5d-4c5846fbb4da None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.511974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf81399-61cb-45bb-8207-851ea842dab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acbcd6961414a07a2fc96da3da3ef0a, user_id=17960d1d84f248fea5646a12bd6185e9, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.512119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef9765-3de1-47da-b999-1c91b11e46a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.512596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef9765-3de1-47da-b999-1c91b11e46a4 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.512876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef9765-3de1-47da-b999-1c91b11e46a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.513141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef9765-3de1-47da-b999-1c91b11e46a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.515164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf81399-61cb-45bb-8207-851ea842dab2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.523206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef9765-3de1-47da-b999-1c91b11e46a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.524602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef9765-3de1-47da-b999-1c91b11e46a4 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.535891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 811/1608] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:32.536505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef9765-3de1-47da-b999-1c91b11e46a4 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.545426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef9765-3de1-47da-b999-1c91b11e46a4 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.552071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdafc9-fe98-4b76-948f-c7d1cb94882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.552959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdafc9-fe98-4b76-948f-c7d1cb94882d admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.555691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdafc9-fe98-4b76-948f-c7d1cb94882d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.555691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdafc9-fe98-4b76-948f-c7d1cb94882d admin admin] PATH_INFO: `/v3/projects/e649b58115ab4bcbbf045a3c036a3049/users/f56fcb6b073d4f1d9dabab1e0bac6ec3/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.570177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdafc9-fe98-4b76-948f-c7d1cb94882d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e649b58115ab4bcbbf045a3c036a3049, user_id=f56fcb6b073d4f1d9dabab1e0bac6ec3, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.573134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdafc9-fe98-4b76-948f-c7d1cb94882d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.617459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 798/1609] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:32 2022] PUT /identity/v3/projects/5acbcd6961414a07a2fc96da3da3ef0a/users/17960d1d84f248fea5646a12bd6185e9/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:32.618296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf81399-61cb-45bb-8207-851ea842dab2 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.674291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf81399-61cb-45bb-8207-851ea842dab2 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.677911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fee18-5edc-4544-b045-01de2ab05e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.678507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fee18-5edc-4544-b045-01de2ab05e3f service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.678779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fee18-5edc-4544-b045-01de2ab05e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.679075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fee18-5edc-4544-b045-01de2ab05e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.714527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 812/1610] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:32 2022] PUT /identity/v3/projects/e649b58115ab4bcbbf045a3c036a3049/users/f56fcb6b073d4f1d9dabab1e0bac6ec3/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:32.715906 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3215b4-4df9-461e-b589-a8f4cb11b475 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.716234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3215b4-4df9-461e-b589-a8f4cb11b475 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.716501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3215b4-4df9-461e-b589-a8f4cb11b475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.736834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7e3215b4-4df9-461e-b589-a8f4cb11b475 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:32.738167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7e3215b4-4df9-461e-b589-a8f4cb11b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:32.743411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fee18-5edc-4544-b045-01de2ab05e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.744858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fee18-5edc-4544-b045-01de2ab05e3f service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.756584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 799/1611] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:32.757348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fee18-5edc-4544-b045-01de2ab05e3f service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.768192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fee18-5edc-4544-b045-01de2ab05e3f service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.804716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 813/1612] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:32.804716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3215b4-4df9-461e-b589-a8f4cb11b475 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:32.838330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f0036-b6b7-4ff8-a58c-745b889422c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.838977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f0036-b6b7-4ff8-a58c-745b889422c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.839242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f0036-b6b7-4ff8-a58c-745b889422c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.839509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f0036-b6b7-4ff8-a58c-745b889422c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.840129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f0036-b6b7-4ff8-a58c-745b889422c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.842029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f0036-b6b7-4ff8-a58c-745b889422c3 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.853493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:32.872328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3215b4-4df9-461e-b589-a8f4cb11b475 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:32.876340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e845f46-fd90-4898-a098-4d773c5c9f62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:32.876969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e845f46-fd90-4898-a098-4d773c5c9f62 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.877243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e845f46-fd90-4898-a098-4d773c5c9f62 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.877675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e845f46-fd90-4898-a098-4d773c5c9f62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.906351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:32.939135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e845f46-fd90-4898-a098-4d773c5c9f62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:32.940740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e845f46-fd90-4898-a098-4d773c5c9f62 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:32.951291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 814/1613] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:32.953249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbed5d7-9082-4c9e-9073-925c7b5639c1 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.953592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbed5d7-9082-4c9e-9073-925c7b5639c1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.955112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbed5d7-9082-4c9e-9073-925c7b5639c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:32.980472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 800/1614] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:32 2022] POST /identity/v3/projects => generated 381 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:32.980472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e0612-aa59-4f49-897d-af2828324c50 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:32.980472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e0612-aa59-4f49-897d-af2828324c50 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:32.980886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e0612-aa59-4f49-897d-af2828324c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.003707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-fbbed5d7-9082-4c9e-9073-925c7b5639c1 None None] MFA Rules not processed for user `17960d1d84f248fea5646a12bd6185e9`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:33.004387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-527e0612-aa59-4f49-897d-af2828324c50 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:33.005222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-fbbed5d7-9082-4c9e-9073-925c7b5639c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:33.006378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-527e0612-aa59-4f49-897d-af2828324c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:33.081658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 801/1615] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:33.082852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e0612-aa59-4f49-897d-af2828324c50 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.093700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-527e0612-aa59-4f49-897d-af2828324c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:33.105292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 815/1616] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:33.106332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbed5d7-9082-4c9e-9073-925c7b5639c1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.120399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbed5d7-9082-4c9e-9073-925c7b5639c1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.126548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845421a7-4fca-47b3-8d4d-2bffd35bbb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.127283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845421a7-4fca-47b3-8d4d-2bffd35bbb4d admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.127565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845421a7-4fca-47b3-8d4d-2bffd35bbb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.127838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845421a7-4fca-47b3-8d4d-2bffd35bbb4d admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.128556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845421a7-4fca-47b3-8d4d-2bffd35bbb4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.131434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845421a7-4fca-47b3-8d4d-2bffd35bbb4d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.183435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e0612-aa59-4f49-897d-af2828324c50 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.187741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946a945-2c27-45d4-9719-2c9a09310f4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.188236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946a945-2c27-45d4-9719-2c9a09310f4a service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.188414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946a945-2c27-45d4-9719-2c9a09310f4a service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.188699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946a945-2c27-45d4-9719-2c9a09310f4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.203844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946a945-2c27-45d4-9719-2c9a09310f4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.210053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946a945-2c27-45d4-9719-2c9a09310f4a service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.218887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 802/1617] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:33.220044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946a945-2c27-45d4-9719-2c9a09310f4a service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.228690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 816/1618] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:33 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:33.232638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845421a7-4fca-47b3-8d4d-2bffd35bbb4d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.251866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845421a7-4fca-47b3-8d4d-2bffd35bbb4d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.252460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946a945-2c27-45d4-9719-2c9a09310f4a service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.260786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996aaef-7f3c-4105-961e-74a6cf2ef79d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.261877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996aaef-7f3c-4105-961e-74a6cf2ef79d service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.261877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996aaef-7f3c-4105-961e-74a6cf2ef79d service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.262826 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996aaef-7f3c-4105-961e-74a6cf2ef79d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.266597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268872c3-1296-4f10-9e28-59ab5a7d6566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.267248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268872c3-1296-4f10-9e28-59ab5a7d6566 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.267551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268872c3-1296-4f10-9e28-59ab5a7d6566 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.267823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268872c3-1296-4f10-9e28-59ab5a7d6566 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.268457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268872c3-1296-4f10-9e28-59ab5a7d6566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.270383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268872c3-1296-4f10-9e28-59ab5a7d6566 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.284254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996aaef-7f3c-4105-961e-74a6cf2ef79d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.286357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996aaef-7f3c-4105-961e-74a6cf2ef79d service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.298034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 803/1619] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:33.303994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996aaef-7f3c-4105-961e-74a6cf2ef79d service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.316062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996aaef-7f3c-4105-961e-74a6cf2ef79d service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.322517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133840b-60ce-4017-a0fa-373bb3356475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.323338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133840b-60ce-4017-a0fa-373bb3356475 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.323687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133840b-60ce-4017-a0fa-373bb3356475 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.324197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133840b-60ce-4017-a0fa-373bb3356475 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.326187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133840b-60ce-4017-a0fa-373bb3356475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.328452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133840b-60ce-4017-a0fa-373bb3356475 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.347282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 804/1620] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:33.350006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133840b-60ce-4017-a0fa-373bb3356475 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.364980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133840b-60ce-4017-a0fa-373bb3356475 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.383488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13853b63-dd7e-4f12-a7dd-c363b2027fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.383488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13853b63-dd7e-4f12-a7dd-c363b2027fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.383488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13853b63-dd7e-4f12-a7dd-c363b2027fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.383488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13853b63-dd7e-4f12-a7dd-c363b2027fdf admin admin] PATH_INFO: `/v3/projects/43ac24e24da0423e82e862c80ee54a2f/users/02e646c52be245b0bd779f05dd229617/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.387766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 817/1621] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:56:33 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:56:33.388895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268872c3-1296-4f10-9e28-59ab5a7d6566 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.402267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13853b63-dd7e-4f12-a7dd-c363b2027fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ac24e24da0423e82e862c80ee54a2f, user_id=02e646c52be245b0bd779f05dd229617, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.411324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13853b63-dd7e-4f12-a7dd-c363b2027fdf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.478694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268872c3-1296-4f10-9e28-59ab5a7d6566 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.484446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2c993-5fbf-4a82-a220-30a2f02b75f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.485120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c993-5fbf-4a82-a220-30a2f02b75f3 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.485444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c993-5fbf-4a82-a220-30a2f02b75f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.485723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c993-5fbf-4a82-a220-30a2f02b75f3 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.486352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2c993-5fbf-4a82-a220-30a2f02b75f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.488426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2c993-5fbf-4a82-a220-30a2f02b75f3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.566522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 805/1622] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:33 2022] PUT /identity/v3/projects/43ac24e24da0423e82e862c80ee54a2f/users/02e646c52be245b0bd779f05dd229617/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:33.566522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13853b63-dd7e-4f12-a7dd-c363b2027fdf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.604738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 818/1623] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:56:33 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:56:33.609932 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2c993-5fbf-4a82-a220-30a2f02b75f3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.628321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2c993-5fbf-4a82-a220-30a2f02b75f3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.640819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:33.646798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:33.669809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13853b63-dd7e-4f12-a7dd-c363b2027fdf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.684833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94304ce2-f995-425c-b43b-c4277507105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.684972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94304ce2-f995-425c-b43b-c4277507105d admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.685585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94304ce2-f995-425c-b43b-c4277507105d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.685585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94304ce2-f995-425c-b43b-c4277507105d admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.686362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94304ce2-f995-425c-b43b-c4277507105d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.688519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94304ce2-f995-425c-b43b-c4277507105d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.701601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 806/1624] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:33.712450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94304ce2-f995-425c-b43b-c4277507105d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.728770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94304ce2-f995-425c-b43b-c4277507105d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.745344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e59189-5fb5-44b6-ac86-286dd1d18d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.745344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e59189-5fb5-44b6-ac86-286dd1d18d87 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.745344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e59189-5fb5-44b6-ac86-286dd1d18d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.745344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e59189-5fb5-44b6-ac86-286dd1d18d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.745682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e59189-5fb5-44b6-ac86-286dd1d18d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.745682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e59189-5fb5-44b6-ac86-286dd1d18d87 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.745838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfb9b7-5ce6-4efc-8e5c-f89ba0533b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.745838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb9b7-5ce6-4efc-8e5c-f89ba0533b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.745838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb9b7-5ce6-4efc-8e5c-f89ba0533b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.745838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb9b7-5ce6-4efc-8e5c-f89ba0533b3d admin admin] PATH_INFO: `/v3/projects/43ac24e24da0423e82e862c80ee54a2f/users/02e646c52be245b0bd779f05dd229617/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.746115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfb9b7-5ce6-4efc-8e5c-f89ba0533b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ac24e24da0423e82e862c80ee54a2f, user_id=02e646c52be245b0bd779f05dd229617, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.750811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfb9b7-5ce6-4efc-8e5c-f89ba0533b3d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.754695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 819/1625] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:56:33.756276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e59189-5fb5-44b6-ac86-286dd1d18d87 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:33.778687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e59189-5fb5-44b6-ac86-286dd1d18d87 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:33.802075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 807/1626] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:33 2022] PUT /identity/v3/projects/43ac24e24da0423e82e862c80ee54a2f/users/02e646c52be245b0bd779f05dd229617/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:33.804543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992955e-e9ce-4d4b-bb8a-1febcba2ec5f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.812754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992955e-e9ce-4d4b-bb8a-1febcba2ec5f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.812754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992955e-e9ce-4d4b-bb8a-1febcba2ec5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.862737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61aa5b4-7c62-4362-9f39-80629339195c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:33.863673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aa5b4-7c62-4362-9f39-80629339195c admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:33.864023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aa5b4-7c62-4362-9f39-80629339195c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:33.864387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61aa5b4-7c62-4362-9f39-80629339195c admin admin] PATH_INFO: `/v3/projects/4053e0b9a658474dad65780e015e84ea/users/93c7cf49da994f608e3baa51326928ed/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:33.867085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7992955e-e9ce-4d4b-bb8a-1febcba2ec5f None None] MFA Rules not processed for user `02e646c52be245b0bd779f05dd229617`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:33.868567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7992955e-e9ce-4d4b-bb8a-1febcba2ec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:33.882815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61aa5b4-7c62-4362-9f39-80629339195c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4053e0b9a658474dad65780e015e84ea, user_id=93c7cf49da994f608e3baa51326928ed, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:33.885859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61aa5b4-7c62-4362-9f39-80629339195c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:33.956178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 808/1627] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:33.971605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992955e-e9ce-4d4b-bb8a-1febcba2ec5f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:34.030043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 820/1628] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:33 2022] PUT /identity/v3/projects/4053e0b9a658474dad65780e015e84ea/users/93c7cf49da994f608e3baa51326928ed/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:34.034177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61aa5b4-7c62-4362-9f39-80629339195c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:34.097574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992955e-e9ce-4d4b-bb8a-1febcba2ec5f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:34.186765 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381e059-77cc-4834-98cc-3c7868f0f792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:34.187875 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381e059-77cc-4834-98cc-3c7868f0f792 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:34.188158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381e059-77cc-4834-98cc-3c7868f0f792 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:34.188520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381e059-77cc-4834-98cc-3c7868f0f792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:34.192609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61aa5b4-7c62-4362-9f39-80629339195c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:34.208751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92c456-2773-4c7c-b00c-78ec8d5a131d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:34.209496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92c456-2773-4c7c-b00c-78ec8d5a131d admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:34.209885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92c456-2773-4c7c-b00c-78ec8d5a131d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:34.210249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92c456-2773-4c7c-b00c-78ec8d5a131d admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:34.211158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92c456-2773-4c7c-b00c-78ec8d5a131d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:34.213051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92c456-2773-4c7c-b00c-78ec8d5a131d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:34.236192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 821/1629] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:56:34.242309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92c456-2773-4c7c-b00c-78ec8d5a131d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:34.262442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92c456-2773-4c7c-b00c-78ec8d5a131d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:34.273102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8715726-6e00-46e0-bb10-de9f326e6cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:34.273933 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8715726-6e00-46e0-bb10-de9f326e6cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:34.274320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8715726-6e00-46e0-bb10-de9f326e6cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:34.274798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8715726-6e00-46e0-bb10-de9f326e6cc7 admin admin] PATH_INFO: `/v3/projects/4053e0b9a658474dad65780e015e84ea/users/93c7cf49da994f608e3baa51326928ed/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:34.278552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8715726-6e00-46e0-bb10-de9f326e6cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4053e0b9a658474dad65780e015e84ea, user_id=93c7cf49da994f608e3baa51326928ed, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:34.278699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381e059-77cc-4834-98cc-3c7868f0f792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:34.279954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381e059-77cc-4834-98cc-3c7868f0f792 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:34.284959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8715726-6e00-46e0-bb10-de9f326e6cc7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:34.303564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 809/1630] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:34.320223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 822/1631] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:56:34 2022] PUT /identity/v3/projects/4053e0b9a658474dad65780e015e84ea/users/93c7cf49da994f608e3baa51326928ed/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:34.332133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7e57e-ca9f-4baa-8497-d769c74e705f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:34.332858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7e57e-ca9f-4baa-8497-d769c74e705f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:34.333315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7e57e-ca9f-4baa-8497-d769c74e705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:34.401389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d4b7e57e-ca9f-4baa-8497-d769c74e705f None None] MFA Rules not processed for user `93c7cf49da994f608e3baa51326928ed`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:34.402847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d4b7e57e-ca9f-4baa-8497-d769c74e705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:34.693205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 810/1632] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:34.698695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8715726-6e00-46e0-bb10-de9f326e6cc7 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:34.698695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:34.698695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8715726-6e00-46e0-bb10-de9f326e6cc7 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:34.698695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ed287-89cf-46ed-ab0d-cd38cc224020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:34.701405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ed287-89cf-46ed-ab0d-cd38cc224020 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:34.701405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ed287-89cf-46ed-ab0d-cd38cc224020 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:34.701405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ed287-89cf-46ed-ab0d-cd38cc224020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:34.701405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ed287-89cf-46ed-ab0d-cd38cc224020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:34.701405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ed287-89cf-46ed-ab0d-cd38cc224020 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:34.701405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 823/1633] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:34.801178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7e57e-ca9f-4baa-8497-d769c74e705f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:34.812300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7e57e-ca9f-4baa-8497-d769c74e705f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:34.818946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e74a4-b00a-4287-abdc-4cfcca9c8f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:34.819574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e74a4-b00a-4287-abdc-4cfcca9c8f5f service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:34.819843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e74a4-b00a-4287-abdc-4cfcca9c8f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:34.820167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e74a4-b00a-4287-abdc-4cfcca9c8f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:34.904938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e74a4-b00a-4287-abdc-4cfcca9c8f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:34.907246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e74a4-b00a-4287-abdc-4cfcca9c8f5f service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:34.913676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:34.922471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 811/1634] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:34.940998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ed287-89cf-46ed-ab0d-cd38cc224020 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:34.951178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ed287-89cf-46ed-ab0d-cd38cc224020 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:34.968837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b6daa-e575-4910-9563-17c41e170090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:34.970970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b6daa-e575-4910-9563-17c41e170090 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:34.970970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b6daa-e575-4910-9563-17c41e170090 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:34.970970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b6daa-e575-4910-9563-17c41e170090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:35.114262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b6daa-e575-4910-9563-17c41e170090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:35.116036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b6daa-e575-4910-9563-17c41e170090 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:35.128071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 824/1635] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:36.291850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0f26-3945-4f4d-9d2b-35be9f2458f1 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:36.292831 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0f26-3945-4f4d-9d2b-35be9f2458f1 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:36.293308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0f26-3945-4f4d-9d2b-35be9f2458f1 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:36.294869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 812/1636] 10.222.0.94 () {54 vars in 962 bytes} [Fri Aug 26 15:56:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:56:36.301383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07b08-e8de-4ed8-91ff-d81df1138a89 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:36.301942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07b08-e8de-4ed8-91ff-d81df1138a89 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:36.301942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07b08-e8de-4ed8-91ff-d81df1138a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:36.394524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a2d07b08-e8de-4ed8-91ff-d81df1138a89 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:36.395824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a2d07b08-e8de-4ed8-91ff-d81df1138a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:36.556786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 825/1637] 10.222.0.94 () {58 vars in 1059 bytes} [Fri Aug 26 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:56:36.562511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4276-b261-45cb-9b73-cdd0b33744e8 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:36.562869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4276-b261-45cb-9b73-cdd0b33744e8 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:36.563107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4276-b261-45cb-9b73-cdd0b33744e8 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:36.564580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 813/1638] 10.222.0.94 () {54 vars in 912 bytes} [Fri Aug 26 15:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:56:36.577041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f9fbe-73a1-4e8d-8738-5019e58932b2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:36.580436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f9fbe-73a1-4e8d-8738-5019e58932b2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:36.581341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f9fbe-73a1-4e8d-8738-5019e58932b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:36.626298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-553f9fbe-73a1-4e8d-8738-5019e58932b2 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:36.627543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-553f9fbe-73a1-4e8d-8738-5019e58932b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:36.644819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 826/1639] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:36.659860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4276-b261-45cb-9b73-cdd0b33744e8 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:36.674734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4276-b261-45cb-9b73-cdd0b33744e8 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:36.682007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484008-ce4a-4c9b-a569-60e8570628dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:36.688496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484008-ce4a-4c9b-a569-60e8570628dd service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:36.688496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484008-ce4a-4c9b-a569-60e8570628dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:36.688496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484008-ce4a-4c9b-a569-60e8570628dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:36.705318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484008-ce4a-4c9b-a569-60e8570628dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:36.707285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484008-ce4a-4c9b-a569-60e8570628dd service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:36.734761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 814/1640] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:39.502778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a2d36-0100-4320-a2a0-32b83356a927 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:39.503308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a2d36-0100-4320-a2a0-32b83356a927 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:39.503393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a2d36-0100-4320-a2a0-32b83356a927 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:39.504697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 827/1641] 10.222.0.94 () {54 vars in 962 bytes} [Fri Aug 26 15:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:56:39.513812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46520802-f43e-4418-b0a5-c90997361f1f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:39.514324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46520802-f43e-4418-b0a5-c90997361f1f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:39.514324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46520802-f43e-4418-b0a5-c90997361f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:39.546856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-46520802-f43e-4418-b0a5-c90997361f1f None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:39.548214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-46520802-f43e-4418-b0a5-c90997361f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:39.575301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 815/1642] 10.222.0.94 () {58 vars in 1059 bytes} [Fri Aug 26 15:56:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:39.581773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf65e63-bc30-46a1-8cfa-a7b0fdf5f3e7 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:39.582122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf65e63-bc30-46a1-8cfa-a7b0fdf5f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:39.582469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf65e63-bc30-46a1-8cfa-a7b0fdf5f3e7 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:39.584663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 828/1643] 10.222.0.94 () {54 vars in 912 bytes} [Fri Aug 26 15:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:56:39.595459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf62ff-34a8-446c-ae1d-8c7368dfa8f9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:39.596586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf62ff-34a8-446c-ae1d-8c7368dfa8f9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:39.599964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf62ff-34a8-446c-ae1d-8c7368dfa8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:39.638770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-bbdf62ff-34a8-446c-ae1d-8c7368dfa8f9 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:39.640268 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bbdf62ff-34a8-446c-ae1d-8c7368dfa8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:39.663612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 816/1644] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 15:56:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:39.673310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf65e63-bc30-46a1-8cfa-a7b0fdf5f3e7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:39.691426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf65e63-bc30-46a1-8cfa-a7b0fdf5f3e7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:39.711187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8e5bb-c921-4630-91c5-15132e579c18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:39.712018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e5bb-c921-4630-91c5-15132e579c18 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:39.715483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e5bb-c921-4630-91c5-15132e579c18 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:39.716023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e5bb-c921-4630-91c5-15132e579c18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:39.751012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8e5bb-c921-4630-91c5-15132e579c18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:39.771715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8e5bb-c921-4630-91c5-15132e579c18 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:39.848039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 829/1645] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:43.649836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aac615-d2f0-4714-9ee2-fe45adb124a7 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.650798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aac615-d2f0-4714-9ee2-fe45adb124a7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.651233 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aac615-d2f0-4714-9ee2-fe45adb124a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.699177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-18aac615-d2f0-4714-9ee2-fe45adb124a7 None None] MFA Rules not processed for user `b50041d36b0b430badd6442a2f09f063`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:43.700429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-18aac615-d2f0-4714-9ee2-fe45adb124a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:43.705473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:43.828853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 817/1646] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:43.861172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8e5bb-c921-4630-91c5-15132e579c18 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:43.882045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8e5bb-c921-4630-91c5-15132e579c18 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:43.901657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6713af-610a-4261-816c-dcf231771150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:43.902703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6713af-610a-4261-816c-dcf231771150 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:43.903157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6713af-610a-4261-816c-dcf231771150 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:43.903610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6713af-610a-4261-816c-dcf231771150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:43.925080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:43.929967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6713af-610a-4261-816c-dcf231771150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:43.938175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6713af-610a-4261-816c-dcf231771150 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:43.975649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 830/1647] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:46.667830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75c0e-354b-47b7-ba6f-25a088ccba85 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:46.668359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75c0e-354b-47b7-ba6f-25a088ccba85 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:46.668359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75c0e-354b-47b7-ba6f-25a088ccba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:46.755390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d4d75c0e-354b-47b7-ba6f-25a088ccba85 None None] MFA Rules not processed for user `7069a9b54a3245b3ab9671730344e6d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:46.757539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d4d75c0e-354b-47b7-ba6f-25a088ccba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:47.065220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 818/1648] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:47.095541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6713af-610a-4261-816c-dcf231771150 service nova] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:47.127485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6713af-610a-4261-816c-dcf231771150 service nova] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:47.152246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df734d9-5196-49d3-ba8e-b5e0e0417cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:47.156642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df734d9-5196-49d3-ba8e-b5e0e0417cfb service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:47.157619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df734d9-5196-49d3-ba8e-b5e0e0417cfb service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:47.159266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df734d9-5196-49d3-ba8e-b5e0e0417cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:47.198174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df734d9-5196-49d3-ba8e-b5e0e0417cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:47.200259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df734d9-5196-49d3-ba8e-b5e0e0417cfb service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:47.222509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 831/1649] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:47.919513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6093a-b99c-45ab-bc8c-63544dea5662 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:47.920028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6093a-b99c-45ab-bc8c-63544dea5662 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:47.920028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6093a-b99c-45ab-bc8c-63544dea5662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:47.996971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-fbd6093a-b99c-45ab-bc8c-63544dea5662 None None] MFA Rules not processed for user `4940f8a9c15a483eab251df392789534`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:47.998265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-fbd6093a-b99c-45ab-bc8c-63544dea5662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:48.215493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 819/1650] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:48.229791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbb88d-adf7-4135-8c9a-6431bcf6b76b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:48.230586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbb88d-adf7-4135-8c9a-6431bcf6b76b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:48.231132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbb88d-adf7-4135-8c9a-6431bcf6b76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:48.270600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e7bbb88d-adf7-4135-8c9a-6431bcf6b76b None None] MFA Rules not processed for user `4940f8a9c15a483eab251df392789534`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:48.272181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e7bbb88d-adf7-4135-8c9a-6431bcf6b76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:48.297056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 832/1651] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:48.313072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6093a-b99c-45ab-bc8c-63544dea5662 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:48.333101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6093a-b99c-45ab-bc8c-63544dea5662 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:48.351154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ad370-4a79-47e3-84f0-a0cdc0ab1168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:48.352232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ad370-4a79-47e3-84f0-a0cdc0ab1168 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:48.352868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ad370-4a79-47e3-84f0-a0cdc0ab1168 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:48.354171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ad370-4a79-47e3-84f0-a0cdc0ab1168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:48.377959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ad370-4a79-47e3-84f0-a0cdc0ab1168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:48.382647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ad370-4a79-47e3-84f0-a0cdc0ab1168 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:48.404883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 820/1652] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:56:49.686003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a165fcf-afd1-427e-8923-5e3f0acf635c None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:49.687521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a165fcf-afd1-427e-8923-5e3f0acf635c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:49.687521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a165fcf-afd1-427e-8923-5e3f0acf635c None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:49.688866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 833/1653] 10.222.0.94 () {54 vars in 912 bytes} [Fri Aug 26 15:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:56:49.698212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f61d9-f8f1-442b-a7bb-8cb7e19eefbe None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:49.698747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f61d9-f8f1-442b-a7bb-8cb7e19eefbe None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:49.698747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f61d9-f8f1-442b-a7bb-8cb7e19eefbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:49.740360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:49.800812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-496f61d9-f8f1-442b-a7bb-8cb7e19eefbe None None] MFA Rules not processed for user `b7ad758153334ec28940156d09495ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:49.806178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-496f61d9-f8f1-442b-a7bb-8cb7e19eefbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:49.943571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 821/1654] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:49.972262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a165fcf-afd1-427e-8923-5e3f0acf635c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:49.994339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a165fcf-afd1-427e-8923-5e3f0acf635c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:49.999423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b8e2c-98a0-4764-891c-14e65fde007e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:50.000619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b8e2c-98a0-4764-891c-14e65fde007e service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:50.001043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b8e2c-98a0-4764-891c-14e65fde007e service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:50.001477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b8e2c-98a0-4764-891c-14e65fde007e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:50.014310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b8e2c-98a0-4764-891c-14e65fde007e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:50.018212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b8e2c-98a0-4764-891c-14e65fde007e service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:50.030711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 834/1655] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:51.933396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f61d9-f8f1-442b-a7bb-8cb7e19eefbe None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:52.178005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f61d9-f8f1-442b-a7bb-8cb7e19eefbe None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:52.189105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a308517-0c97-4adb-b886-846fb05d38b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:52.190050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308517-0c97-4adb-b886-846fb05d38b4 service placement] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:52.190679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308517-0c97-4adb-b886-846fb05d38b4 service placement] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:52.191210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308517-0c97-4adb-b886-846fb05d38b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:52.193286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6a308517-0c97-4adb-b886-846fb05d38b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:52.219557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a308517-0c97-4adb-b886-846fb05d38b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:52.225072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a308517-0c97-4adb-b886-846fb05d38b4 service placement] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:52.270222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 822/1656] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:52.281744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fab15-428d-4efa-8397-55acff9f9a2e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:52.282367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fab15-428d-4efa-8397-55acff9f9a2e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:52.282367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fab15-428d-4efa-8397-55acff9f9a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:52.323468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-953fab15-428d-4efa-8397-55acff9f9a2e None None] MFA Rules not processed for user `f56fcb6b073d4f1d9dabab1e0bac6ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:52.324800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-953fab15-428d-4efa-8397-55acff9f9a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:52.354397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bff0c-3901-4863-9c20-8bf0e4cf20af None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:52.354942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bff0c-3901-4863-9c20-8bf0e4cf20af None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:52.355359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bff0c-3901-4863-9c20-8bf0e4cf20af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:52.393832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-1a9bff0c-3901-4863-9c20-8bf0e4cf20af None None] MFA Rules not processed for user `4940f8a9c15a483eab251df392789534`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:52.403568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1a9bff0c-3901-4863-9c20-8bf0e4cf20af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:52.423582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 823/1657] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:56:52.497743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca61c9-cb92-4564-87b2-4d4037633ceb None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:52.498205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca61c9-cb92-4564-87b2-4d4037633ceb None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:52.499204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca61c9-cb92-4564-87b2-4d4037633ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:52.501485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 835/1658] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:52.557662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-04ca61c9-cb92-4564-87b2-4d4037633ceb None None] MFA Rules not processed for user `f56fcb6b073d4f1d9dabab1e0bac6ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:52.567034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-04ca61c9-cb92-4564-87b2-4d4037633ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:52.603421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 824/1659] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:56:52.612624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fab15-428d-4efa-8397-55acff9f9a2e None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:52.645982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fab15-428d-4efa-8397-55acff9f9a2e None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:52.662540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c2e8b-b832-48b3-af82-37ac09fa068d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:52.663872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c2e8b-b832-48b3-af82-37ac09fa068d service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:52.665197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c2e8b-b832-48b3-af82-37ac09fa068d service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:52.665503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c2e8b-b832-48b3-af82-37ac09fa068d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:52.693555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c2e8b-b832-48b3-af82-37ac09fa068d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:52.695806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c2e8b-b832-48b3-af82-37ac09fa068d service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:52.729705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 836/1660] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:53.429047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fbde0-23cd-4810-9ba7-65a306ae364d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:53.429622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fbde0-23cd-4810-9ba7-65a306ae364d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:53.429622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fbde0-23cd-4810-9ba7-65a306ae364d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:53.465776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-378fbde0-23cd-4810-9ba7-65a306ae364d None None] MFA Rules not processed for user `228837054cd648acb0df93dd423e1d83`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:53.468218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-378fbde0-23cd-4810-9ba7-65a306ae364d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:53.598073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 825/1661] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:53.757139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd3c0-82ef-4496-a2d3-f795cf8b0434 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:53.757673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd3c0-82ef-4496-a2d3-f795cf8b0434 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:53.757673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd3c0-82ef-4496-a2d3-f795cf8b0434 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:53.759830 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 837/1662] 10.222.0.94 () {54 vars in 912 bytes} [Fri Aug 26 15:56:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:56:53.768388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfea20c-a884-4dcc-a553-1a28485e4a79 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:53.768867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfea20c-a884-4dcc-a553-1a28485e4a79 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:53.769330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfea20c-a884-4dcc-a553-1a28485e4a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:53.789839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4dfea20c-a884-4dcc-a553-1a28485e4a79 None None] MFA Rules not processed for user `b7ad758153334ec28940156d09495ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:53.791177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4dfea20c-a884-4dcc-a553-1a28485e4a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:53.799870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 826/1663] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 15:56:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:53.847677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd3c0-82ef-4496-a2d3-f795cf8b0434 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:53.860583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd3c0-82ef-4496-a2d3-f795cf8b0434 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:53.868347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e8897-ec57-4dfc-9c28-5ccdb90dcd98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:53.868971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8897-ec57-4dfc-9c28-5ccdb90dcd98 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:53.869517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8897-ec57-4dfc-9c28-5ccdb90dcd98 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:53.870079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8897-ec57-4dfc-9c28-5ccdb90dcd98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:53.892470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8897-ec57-4dfc-9c28-5ccdb90dcd98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:53.894733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8897-ec57-4dfc-9c28-5ccdb90dcd98 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:53.904143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 838/1664] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:54.893943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfea20c-a884-4dcc-a553-1a28485e4a79 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:54.940186 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfea20c-a884-4dcc-a553-1a28485e4a79 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:54.974162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b3435-b2f7-4a65-8da0-8c0297097a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:54.982841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b3435-b2f7-4a65-8da0-8c0297097a73 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:54.982841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b3435-b2f7-4a65-8da0-8c0297097a73 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:54.982841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b3435-b2f7-4a65-8da0-8c0297097a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.001115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:55.011837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b3435-b2f7-4a65-8da0-8c0297097a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:55.019921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b3435-b2f7-4a65-8da0-8c0297097a73 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:55.036146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 827/1665] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:55.222424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a0916-23a3-4158-8109-826d30341f20 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:55.224981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a0916-23a3-4158-8109-826d30341f20 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:55.225540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a0916-23a3-4158-8109-826d30341f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:55.268792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-096a0916-23a3-4158-8109-826d30341f20 None None] MFA Rules not processed for user `b50041d36b0b430badd6442a2f09f063`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:55.270460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-096a0916-23a3-4158-8109-826d30341f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:55.285619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 839/1666] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:55.965083 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b3435-b2f7-4a65-8da0-8c0297097a73 service nova] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:56.206646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b3435-b2f7-4a65-8da0-8c0297097a73 service nova] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:56.212849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:56.213769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:56.214230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:56.214679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] PATH_INFO: `/v3/users/80f71a262e95480582c37cd9541b41ba` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:56.216701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f71a262e95480582c37cd9541b41ba)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:56.220335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:56.483942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f71a262e95480582c37cd9541b41ba'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:56.487793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f71a262e95480582c37cd9541b41ba'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:56.541470 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f71a262e95480582c37cd9541b41ba'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:56.541470 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f71a262e95480582c37cd9541b41ba'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:56.742143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f71a262e95480582c37cd9541b41ba'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:56.769659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 828/1667] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:55 2022] DELETE /identity/v3/users/80f71a262e95480582c37cd9541b41ba => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:56.770807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa86cb-a5ab-4f81-ad0e-7a91e316f229 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:56.771394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa86cb-a5ab-4f81-ad0e-7a91e316f229 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:56.771394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa86cb-a5ab-4f81-ad0e-7a91e316f229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:56.815148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-63fa86cb-a5ab-4f81-ad0e-7a91e316f229 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:56.816988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-63fa86cb-a5ab-4f81-ad0e-7a91e316f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:56.903633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 840/1668] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:56.917851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:56.924470 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:57.047975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616cc7f-55b6-42e1-8bba-dc20203484aa None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.048346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616cc7f-55b6-42e1-8bba-dc20203484aa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.048609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616cc7f-55b6-42e1-8bba-dc20203484aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.093090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-c616cc7f-55b6-42e1-8bba-dc20203484aa None None] MFA Rules not processed for user `7069a9b54a3245b3ab9671730344e6d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:57.098162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c616cc7f-55b6-42e1-8bba-dc20203484aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:57.169002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:57.185631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc813e-a95e-41b9-9dba-cc8f7ed23877 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:57.192112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:57.194031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.194563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.195079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.212939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:57.215445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:57.256030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 829/1669] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:57.294193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:57.298682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:57.324993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2837ba-cbf4-4ff7-a716-cf6fda465049 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:57.329947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:57.332177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.332731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.333453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] PATH_INFO: `/v3/users/bf6426addc5f4cb49456968c16e2f370` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.335082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6426addc5f4cb49456968c16e2f370)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:57.340048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:57.373184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 841/1670] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:57.373279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:57.676070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6426addc5f4cb49456968c16e2f370'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.676772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6426addc5f4cb49456968c16e2f370'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.711627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6426addc5f4cb49456968c16e2f370'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.714812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6426addc5f4cb49456968c16e2f370'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.799677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6426addc5f4cb49456968c16e2f370'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:57.822580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 830/1671] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:57 2022] DELETE /identity/v3/users/bf6426addc5f4cb49456968c16e2f370 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:57.827753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dcfd7-d844-4722-a318-68098e8eea53 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:57.828658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dcfd7-d844-4722-a318-68098e8eea53 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:57.828658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dcfd7-d844-4722-a318-68098e8eea53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:57.874658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-bb5dcfd7-d844-4722-a318-68098e8eea53 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:56:57.876365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-bb5dcfd7-d844-4722-a318-68098e8eea53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:58.032470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 842/1672] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:56:58.078879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:58.083852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:58.242631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892747f-72c3-4c48-8db7-23f5a1e0b8bb admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:58.249698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47d5a5-7c35-43d1-9d36-5bb630aa610b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:58.250518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d5a5-7c35-43d1-9d36-5bb630aa610b service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:58.250949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d5a5-7c35-43d1-9d36-5bb630aa610b service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:58.251353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d5a5-7c35-43d1-9d36-5bb630aa610b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:58.288013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47d5a5-7c35-43d1-9d36-5bb630aa610b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:58.289804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47d5a5-7c35-43d1-9d36-5bb630aa610b service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:58.325030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 831/1673] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:56:58.372516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dcfd7-d844-4722-a318-68098e8eea53 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:58.375296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-bb5dcfd7-d844-4722-a318-68098e8eea53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:58.409768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dcfd7-d844-4722-a318-68098e8eea53 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:58.441412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:58.442095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:58.442368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:58.442638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] PATH_INFO: `/v3/projects/0d8be83798d04491b2e420af8502106f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:58.447039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8be83798d04491b2e420af8502106f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:58.449245 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:58.778056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8be83798d04491b2e420af8502106f', 'request_id': 'req-59625634-217b-4831-85ac-5a6ee469ecca'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.828789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8be83798d04491b2e420af8502106f', 'request_id': 'req-59625634-217b-4831-85ac-5a6ee469ecca'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.942901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8be83798d04491b2e420af8502106f', 'request_id': 'req-59625634-217b-4831-85ac-5a6ee469ecca'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.959550 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] The token cache is being invalidate because project 0d8be83798d04491b2e420af8502106f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:56:58.959840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.964982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:58.967954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 843/1674] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:56:58 2022] DELETE /identity/v3/projects/0d8be83798d04491b2e420af8502106f => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:58.972495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47d5a5-7c35-43d1-9d36-5bb630aa610b service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:59.030795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47d5a5-7c35-43d1-9d36-5bb630aa610b service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:59.244339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:59.246039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:59.246678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:59.247278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] PATH_INFO: `/v3/users/02e646c52be245b0bd779f05dd229617` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:59.254012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02e646c52be245b0bd779f05dd229617)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:59.285526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:59.485158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:56:59.485158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:56:59.527531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59625634-217b-4831-85ac-5a6ee469ecca admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:56:59.534809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:56:59.536281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:56:59.537408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:56:59.537890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] PATH_INFO: `/v3/projects/d6f7c7361aad41a594a0164ebfac2a0c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:56:59.542898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f7c7361aad41a594a0164ebfac2a0c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:56:59.548473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:56:59.569717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02e646c52be245b0bd779f05dd229617'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.571671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02e646c52be245b0bd779f05dd229617'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.605312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02e646c52be245b0bd779f05dd229617'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.606079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02e646c52be245b0bd779f05dd229617'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.684510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bbb5297f-5e24-44c5-a484-277f68b15d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02e646c52be245b0bd779f05dd229617'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.712940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 832/1675] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:56:58 2022] DELETE /identity/v3/users/02e646c52be245b0bd779f05dd229617 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:56:59.802678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:59.811671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f7c7361aad41a594a0164ebfac2a0c', 'request_id': 'req-3116ae06-0b5b-413a-8566-f74852fc3a00'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.832175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f7c7361aad41a594a0164ebfac2a0c', 'request_id': 'req-3116ae06-0b5b-413a-8566-f74852fc3a00'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:56:59.873775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:56:59.986996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f7c7361aad41a594a0164ebfac2a0c', 'request_id': 'req-3116ae06-0b5b-413a-8566-f74852fc3a00'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.003761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] The token cache is being invalidate because project d6f7c7361aad41a594a0164ebfac2a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:00.004400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.005206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3116ae06-0b5b-413a-8566-f74852fc3a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:00.006896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 844/1676] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:56:59 2022] DELETE /identity/v3/projects/d6f7c7361aad41a594a0164ebfac2a0c => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:00.012056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d7deb-01fb-492a-9e8a-b8496c8ac287 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.012426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d7deb-01fb-492a-9e8a-b8496c8ac287 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.012762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d7deb-01fb-492a-9e8a-b8496c8ac287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.049998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ef8d7deb-01fb-492a-9e8a-b8496c8ac287 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:00.051447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ef8d7deb-01fb-492a-9e8a-b8496c8ac287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:00.203076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 833/1677] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:00.207744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74a24f-4fbb-44c0-9894-c140b1a2d781 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.208224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74a24f-4fbb-44c0-9894-c140b1a2d781 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.208758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74a24f-4fbb-44c0-9894-c140b1a2d781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.247084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2c74a24f-4fbb-44c0-9894-c140b1a2d781 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:00.252447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2c74a24f-4fbb-44c0-9894-c140b1a2d781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:00.276887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 845/1678] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:00.277024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d7deb-01fb-492a-9e8a-b8496c8ac287 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.303840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d7deb-01fb-492a-9e8a-b8496c8ac287 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.327958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.329480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.329756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.330022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.339427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:00.339758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.344741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.383295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 834/1679] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:00.387669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74a24f-4fbb-44c0-9894-c140b1a2d781 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.404425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74a24f-4fbb-44c0-9894-c140b1a2d781 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.424223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a4cdf-81b0-4054-aa95-896f4ad15e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.425857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4cdf-81b0-4054-aa95-896f4ad15e30 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.426276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4cdf-81b0-4054-aa95-896f4ad15e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.427327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4cdf-81b0-4054-aa95-896f4ad15e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.429025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a4cdf-81b0-4054-aa95-896f4ad15e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.433535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a4cdf-81b0-4054-aa95-896f4ad15e30 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.621342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 846/1680] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/projects => generated 405 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:00.624520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:00.645224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4e98e-8cd1-4577-81f6-4f88e6562d03 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:00.776505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192673a-d03a-40a9-9dc9-d8f6e2a896e8 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.776606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192673a-d03a-40a9-9dc9-d8f6e2a896e8 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.776886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192673a-d03a-40a9-9dc9-d8f6e2a896e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.783867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e44c6-131b-4f1e-9b05-460fef70abec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:00.794092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e44c6-131b-4f1e-9b05-460fef70abec admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:00.794574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e44c6-131b-4f1e-9b05-460fef70abec admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:00.796042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e44c6-131b-4f1e-9b05-460fef70abec admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:00.797625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e44c6-131b-4f1e-9b05-460fef70abec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:00.804934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e44c6-131b-4f1e-9b05-460fef70abec admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:00.818160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6192673a-d03a-40a9-9dc9-d8f6e2a896e8 None None] MFA Rules not processed for user `228837054cd648acb0df93dd423e1d83`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:00.819206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6192673a-d03a-40a9-9dc9-d8f6e2a896e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:00.968462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 847/1681] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:01.012066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 835/1682] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:00 2022] POST /identity/v3/users => generated 303 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:01.018003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192673a-d03a-40a9-9dc9-d8f6e2a896e8 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.073219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192673a-d03a-40a9-9dc9-d8f6e2a896e8 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.086573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d41519-3c31-4efb-8f32-3b9da29e2eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.088214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41519-3c31-4efb-8f32-3b9da29e2eda admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.088705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41519-3c31-4efb-8f32-3b9da29e2eda admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.089295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41519-3c31-4efb-8f32-3b9da29e2eda admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.090230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d41519-3c31-4efb-8f32-3b9da29e2eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.092178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d41519-3c31-4efb-8f32-3b9da29e2eda admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.112447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 848/1683] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:01.116102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e44c6-131b-4f1e-9b05-460fef70abec admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.145777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e44c6-131b-4f1e-9b05-460fef70abec admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.170010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ecdac0-73e5-4878-96f3-676562f9bf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.171566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecdac0-73e5-4878-96f3-676562f9bf3c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.172191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecdac0-73e5-4878-96f3-676562f9bf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.172686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecdac0-73e5-4878-96f3-676562f9bf3c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.173544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ecdac0-73e5-4878-96f3-676562f9bf3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.177665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ecdac0-73e5-4878-96f3-676562f9bf3c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.193695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 836/1684] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:01.196768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d41519-3c31-4efb-8f32-3b9da29e2eda admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.215387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d41519-3c31-4efb-8f32-3b9da29e2eda admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.231952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf908c3-f547-483e-a160-048b1b26b837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.232905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf908c3-f547-483e-a160-048b1b26b837 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.233283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf908c3-f547-483e-a160-048b1b26b837 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.233678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf908c3-f547-483e-a160-048b1b26b837 admin admin] PATH_INFO: `/v3/projects/65bea261cb0c4ed39a65758ca8def241/users/2b27283f35d54b45b01b1da900eb2f3a/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.272855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa10a0-196f-48ba-bf03-229ea4206041 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.273396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa10a0-196f-48ba-bf03-229ea4206041 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.273862 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa10a0-196f-48ba-bf03-229ea4206041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.292795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf908c3-f547-483e-a160-048b1b26b837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65bea261cb0c4ed39a65758ca8def241, user_id=2b27283f35d54b45b01b1da900eb2f3a, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.302603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf908c3-f547-483e-a160-048b1b26b837 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.390752 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-05aa10a0-196f-48ba-bf03-229ea4206041 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:01.392303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-05aa10a0-196f-48ba-bf03-229ea4206041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:01.422973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 849/1685] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:01 2022] PUT /identity/v3/projects/65bea261cb0c4ed39a65758ca8def241/users/2b27283f35d54b45b01b1da900eb2f3a/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:01.424889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909f4a-fed4-4728-b988-dfd919e8c5e1 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.425235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909f4a-fed4-4728-b988-dfd919e8c5e1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.425511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909f4a-fed4-4728-b988-dfd919e8c5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.476731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1a909f4a-fed4-4728-b988-dfd919e8c5e1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:01.478308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1a909f4a-fed4-4728-b988-dfd919e8c5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:01.607133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 837/1686] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:01.625355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa10a0-196f-48ba-bf03-229ea4206041 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.630058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-05aa10a0-196f-48ba-bf03-229ea4206041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:01.685338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa10a0-196f-48ba-bf03-229ea4206041 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:01.729379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 850/1687] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:01.893760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a909f4a-fed4-4728-b988-dfd919e8c5e1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:01.895411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1a909f4a-fed4-4728-b988-dfd919e8c5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:01.943755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:01.944660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:01.946235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:01.947676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] PATH_INFO: `/v3/projects/43ac24e24da0423e82e862c80ee54a2f` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:01.953984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ac24e24da0423e82e862c80ee54a2f)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:01.959581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: WARNING py.warnings [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:57:01.959581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: warnings.warn(msg) Aug 26 15:57:01.959581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: Aug 26 15:57:01.963222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:01.997034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a909f4a-fed4-4728-b988-dfd919e8c5e1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:02.002733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:02.003525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.003929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.004390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.019576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:02.021222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:02.054206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 851/1688] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:02.182316 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:02.208243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:02.445274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ac24e24da0423e82e862c80ee54a2f', 'request_id': 'req-24ec8c80-5902-4acf-bd6f-de0cb58e051f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:02.474823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ac24e24da0423e82e862c80ee54a2f', 'request_id': 'req-24ec8c80-5902-4acf-bd6f-de0cb58e051f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:02.592026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144b4f8-87d0-4608-b5b9-bd7808c2a097 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:02.622035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ac24e24da0423e82e862c80ee54a2f', 'request_id': 'req-24ec8c80-5902-4acf-bd6f-de0cb58e051f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:02.678019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] The token cache is being invalidate because project 43ac24e24da0423e82e862c80ee54a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:02.682037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:02.683101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24ec8c80-5902-4acf-bd6f-de0cb58e051f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:02.700576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 838/1689] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:01 2022] DELETE /identity/v3/projects/43ac24e24da0423e82e862c80ee54a2f => generated 0 bytes in 1077 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:02.700576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72360d-16d9-40c9-ae1d-2243c63efd31 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.700576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72360d-16d9-40c9-ae1d-2243c63efd31 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.700576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72360d-16d9-40c9-ae1d-2243c63efd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.735612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:02.736259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.736698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.736819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.739442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:02.746964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-1b72360d-16d9-40c9-ae1d-2243c63efd31 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:02.750310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1b72360d-16d9-40c9-ae1d-2243c63efd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:02.775808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:02.895759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 839/1690] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:02.905262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97dee-4bd5-47ab-91d6-8fb6063808bb None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:02.905559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97dee-4bd5-47ab-91d6-8fb6063808bb None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:02.905939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97dee-4bd5-47ab-91d6-8fb6063808bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:02.944044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:02.950427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:02.950579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8cc97dee-4bd5-47ab-91d6-8fb6063808bb None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:02.950579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8cc97dee-4bd5-47ab-91d6-8fb6063808bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:02.968871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 840/1691] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:02.971104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 852/1692] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 15:57:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 791 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 15:57:02.972626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc97dee-4bd5-47ab-91d6-8fb6063808bb None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:02.991028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc97dee-4bd5-47ab-91d6-8fb6063808bb None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:03.008117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:03.008398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:03.008688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:03.008933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:03.009615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:03.009916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:03.011616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:03.017286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:03.071990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 841/1693] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:03.075093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:03.095292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cd670-142a-4827-b705-5136fc369950 service swift] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:03.126159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f117c-657d-4274-b98a-e4a61961ba4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:03.127119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f117c-657d-4274-b98a-e4a61961ba4c admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:03.127588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f117c-657d-4274-b98a-e4a61961ba4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:03.128022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f117c-657d-4274-b98a-e4a61961ba4c admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:03.128832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f117c-657d-4274-b98a-e4a61961ba4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:03.130988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f117c-657d-4274-b98a-e4a61961ba4c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:03.293654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 853/1694] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:03 2022] POST /identity/v3/projects => generated 397 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:03.299405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:03.317999 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7b042-9ba6-4ea9-a43e-9c8591124888 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:03.470474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156118c-7bf9-4771-84ad-3b5f6792baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:03.473955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156118c-7bf9-4771-84ad-3b5f6792baab admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:03.475286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156118c-7bf9-4771-84ad-3b5f6792baab admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:03.475564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156118c-7bf9-4771-84ad-3b5f6792baab admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:03.477873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156118c-7bf9-4771-84ad-3b5f6792baab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:03.486745 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156118c-7bf9-4771-84ad-3b5f6792baab admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:03.649150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 842/1695] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:03 2022] POST /identity/v3/users => generated 299 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:03.650092 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f117c-657d-4274-b98a-e4a61961ba4c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:03.669847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f117c-657d-4274-b98a-e4a61961ba4c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:03.684077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d7f8-1301-458f-bd0e-0187e119257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:03.685213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d7f8-1301-458f-bd0e-0187e119257a admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:03.687511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d7f8-1301-458f-bd0e-0187e119257a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:03.687989 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d7f8-1301-458f-bd0e-0187e119257a admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:03.691839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106d7f8-1301-458f-bd0e-0187e119257a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:03.695515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106d7f8-1301-458f-bd0e-0187e119257a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:03.754668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 854/1696] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:03.759192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156118c-7bf9-4771-84ad-3b5f6792baab admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:03.775226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156118c-7bf9-4771-84ad-3b5f6792baab admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:03.791724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50baeaf5-76bc-4238-b070-fd12f1d62b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:03.792359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baeaf5-76bc-4238-b070-fd12f1d62b26 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:03.792667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baeaf5-76bc-4238-b070-fd12f1d62b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:03.792910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50baeaf5-76bc-4238-b070-fd12f1d62b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:03.797848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50baeaf5-76bc-4238-b070-fd12f1d62b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:03.803242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50baeaf5-76bc-4238-b070-fd12f1d62b26 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:03.852687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 843/1697] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:03.862459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d7f8-1301-458f-bd0e-0187e119257a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:03.876477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d7f8-1301-458f-bd0e-0187e119257a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:03.897960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540069-13fb-4e4c-8a4a-914be79c8c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:03.899103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540069-13fb-4e4c-8a4a-914be79c8c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:03.899870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540069-13fb-4e4c-8a4a-914be79c8c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:03.901463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540069-13fb-4e4c-8a4a-914be79c8c4c admin admin] PATH_INFO: `/v3/projects/7279711958be4782a08f82bcfa02c2ab/users/1810de94548541329acb0284420ca11a/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:03.934034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540069-13fb-4e4c-8a4a-914be79c8c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7279711958be4782a08f82bcfa02c2ab, user_id=1810de94548541329acb0284420ca11a, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:03.965174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540069-13fb-4e4c-8a4a-914be79c8c4c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:04.058622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 855/1698] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:03 2022] PUT /identity/v3/projects/7279711958be4782a08f82bcfa02c2ab/users/1810de94548541329acb0284420ca11a/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:04.075772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b8c8-f9c0-4529-9e3c-0ca7dba601aa None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:04.076056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b8c8-f9c0-4529-9e3c-0ca7dba601aa None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:04.076420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3b8c8-f9c0-4529-9e3c-0ca7dba601aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:04.132697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-9db3b8c8-f9c0-4529-9e3c-0ca7dba601aa None None] MFA Rules not processed for user `1810de94548541329acb0284420ca11a`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:04.133994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9db3b8c8-f9c0-4529-9e3c-0ca7dba601aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:04.411366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 844/1699] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:04.415490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540069-13fb-4e4c-8a4a-914be79c8c4c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:04.487478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee8ebb-ddf3-41f0-a697-bde9a3350d8e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:04.488594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee8ebb-ddf3-41f0-a697-bde9a3350d8e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:04.489019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee8ebb-ddf3-41f0-a697-bde9a3350d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:04.575174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-2bee8ebb-ddf3-41f0-a697-bde9a3350d8e None None] MFA Rules not processed for user `b7ad758153334ec28940156d09495ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:04.576981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2bee8ebb-ddf3-41f0-a697-bde9a3350d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:04.626657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540069-13fb-4e4c-8a4a-914be79c8c4c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:04.641891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9c160-2bf7-45bc-babd-ce178989de48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:04.642355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9c160-2bf7-45bc-babd-ce178989de48 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:04.642632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9c160-2bf7-45bc-babd-ce178989de48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:04.642907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9c160-2bf7-45bc-babd-ce178989de48 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:04.643562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9c160-2bf7-45bc-babd-ce178989de48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:04.646041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9c160-2bf7-45bc-babd-ce178989de48 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:04.764857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 845/1700] 10.222.0.102 () {58 vars in 1016 bytes} [Fri Aug 26 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:04.801362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee8ebb-ddf3-41f0-a697-bde9a3350d8e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:04.887815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2bee8ebb-ddf3-41f0-a697-bde9a3350d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:04.887815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee8ebb-ddf3-41f0-a697-bde9a3350d8e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:04.887815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b62f-eecd-4bb3-9373-e7c79df2a7f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:04.889431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b62f-eecd-4bb3-9373-e7c79df2a7f9 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:04.889431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b62f-eecd-4bb3-9373-e7c79df2a7f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:04.889431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b62f-eecd-4bb3-9373-e7c79df2a7f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:04.889431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084b62f-eecd-4bb3-9373-e7c79df2a7f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:04.889431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084b62f-eecd-4bb3-9373-e7c79df2a7f9 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:04.889431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 846/1701] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:05.106739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 856/1702] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:04 2022] POST /identity/v3/projects => generated 401 bytes in 692 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:05.112635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b62f-eecd-4bb3-9373-e7c79df2a7f9 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:05.130137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b62f-eecd-4bb3-9373-e7c79df2a7f9 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:05.256889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470fbd-aaf7-4d14-98ec-c5dbc4552783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:05.257670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470fbd-aaf7-4d14-98ec-c5dbc4552783 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:05.258969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470fbd-aaf7-4d14-98ec-c5dbc4552783 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:05.258969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470fbd-aaf7-4d14-98ec-c5dbc4552783 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:05.259755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c470fbd-aaf7-4d14-98ec-c5dbc4552783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:05.262223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c470fbd-aaf7-4d14-98ec-c5dbc4552783 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:05.464734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 847/1703] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:05 2022] POST /identity/v3/users => generated 307 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:05.469471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9c160-2bf7-45bc-babd-ce178989de48 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:05.490148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9c160-2bf7-45bc-babd-ce178989de48 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:05.531954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cc9fc-d309-4818-93ed-e939794febe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:05.532662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc9fc-d309-4818-93ed-e939794febe4 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:05.533045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc9fc-d309-4818-93ed-e939794febe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:05.533547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc9fc-d309-4818-93ed-e939794febe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:05.534425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cc9fc-d309-4818-93ed-e939794febe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:05.536529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cc9fc-d309-4818-93ed-e939794febe4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:05.593025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 857/1704] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:05.597629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470fbd-aaf7-4d14-98ec-c5dbc4552783 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:05.599504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:05.629054 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470fbd-aaf7-4d14-98ec-c5dbc4552783 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:05.639056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48c646-fb16-4a87-87df-6149cbc441ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:05.645121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c646-fb16-4a87-87df-6149cbc441ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:05.645121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c646-fb16-4a87-87df-6149cbc441ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:05.645121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c646-fb16-4a87-87df-6149cbc441ec admin admin] PATH_INFO: `/v3/projects/f053226c8ae94f3c8c602ac7a0187d15/users/69c5bee0c3eb4f4896df20a0d74334ee/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:05.698080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48c646-fb16-4a87-87df-6149cbc441ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f053226c8ae94f3c8c602ac7a0187d15, user_id=69c5bee0c3eb4f4896df20a0d74334ee, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:05.720290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:05.721217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48c646-fb16-4a87-87df-6149cbc441ec admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:05.840202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 848/1705] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:05 2022] PUT /identity/v3/projects/f053226c8ae94f3c8c602ac7a0187d15/users/69c5bee0c3eb4f4896df20a0d74334ee/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:05.845616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cc9fc-d309-4818-93ed-e939794febe4 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:05.861546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cc9fc-d309-4818-93ed-e939794febe4 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:06.046372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495f43d-bc3a-4319-8086-9a482a707a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:06.046372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f43d-bc3a-4319-8086-9a482a707a45 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:06.046701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f43d-bc3a-4319-8086-9a482a707a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:06.046980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f43d-bc3a-4319-8086-9a482a707a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:06.047637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495f43d-bc3a-4319-8086-9a482a707a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:06.060705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495f43d-bc3a-4319-8086-9a482a707a45 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:06.085066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 858/1706] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:06.096245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48c646-fb16-4a87-87df-6149cbc441ec admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:06.116904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48c646-fb16-4a87-87df-6149cbc441ec admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:06.137603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:06.138504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:06.139420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:06.142420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] PATH_INFO: `/v3/projects/f053226c8ae94f3c8c602ac7a0187d15/users/69c5bee0c3eb4f4896df20a0d74334ee/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:06.147701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f053226c8ae94f3c8c602ac7a0187d15, user_id=69c5bee0c3eb4f4896df20a0d74334ee, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:06.158854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:06.213073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 849/1707] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:06 2022] PUT /identity/v3/projects/f053226c8ae94f3c8c602ac7a0187d15/users/69c5bee0c3eb4f4896df20a0d74334ee/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:06.225698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08614-99cf-4093-9e6c-668573a7d78c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:06.226251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08614-99cf-4093-9e6c-668573a7d78c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:06.226741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08614-99cf-4093-9e6c-668573a7d78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:06.310096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-dab08614-99cf-4093-9e6c-668573a7d78c None None] MFA Rules not processed for user `69c5bee0c3eb4f4896df20a0d74334ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:06.321487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-dab08614-99cf-4093-9e6c-668573a7d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:06.559299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 859/1708] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:06 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:06.573097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:06.573354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:06.585751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:06.772554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d19826-a546-4770-9d5d-229d14942f0e admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:06.783399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4314d-12c8-44e9-9db7-d5f0c283e39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:06.784476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4314d-12c8-44e9-9db7-d5f0c283e39c service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:06.784912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4314d-12c8-44e9-9db7-d5f0c283e39c service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:06.785385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4314d-12c8-44e9-9db7-d5f0c283e39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:06.809147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4314d-12c8-44e9-9db7-d5f0c283e39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:06.811524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4314d-12c8-44e9-9db7-d5f0c283e39c service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:06.839433 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 850/1709] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:06 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:07.129241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab08614-99cf-4093-9e6c-668573a7d78c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:07.148454 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab08614-99cf-4093-9e6c-668573a7d78c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:07.156158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32467be3-67b6-4f04-9a61-d9c87f2a669f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:07.167588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32467be3-67b6-4f04-9a61-d9c87f2a669f service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:07.167588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32467be3-67b6-4f04-9a61-d9c87f2a669f service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:07.167588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32467be3-67b6-4f04-9a61-d9c87f2a669f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:07.322296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32467be3-67b6-4f04-9a61-d9c87f2a669f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:07.324214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32467be3-67b6-4f04-9a61-d9c87f2a669f service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:07.336230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 860/1710] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:08.923257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:09.429102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e33e7d-ce0e-45e3-a232-be59468f6d22 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:09.429825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e33e7d-ce0e-45e3-a232-be59468f6d22 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:09.430693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e33e7d-ce0e-45e3-a232-be59468f6d22 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:09.433083 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 851/1711] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 15:57:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:57:09.468006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587087a-27f1-4eca-a06d-6d487f09ca21 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:09.471654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587087a-27f1-4eca-a06d-6d487f09ca21 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:09.473138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587087a-27f1-4eca-a06d-6d487f09ca21 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:09.476569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 861/1712] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 15:57:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:57:09.594352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26c878-9dfe-49ef-bbeb-03cb52c6fa79 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:09.595237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26c878-9dfe-49ef-bbeb-03cb52c6fa79 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:09.595828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26c878-9dfe-49ef-bbeb-03cb52c6fa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:09.688966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-2d26c878-9dfe-49ef-bbeb-03cb52c6fa79 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:09.702063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2d26c878-9dfe-49ef-bbeb-03cb52c6fa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:09.784177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c35e73-081d-44dd-8867-1b2f70aeec19 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:09.784680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c35e73-081d-44dd-8867-1b2f70aeec19 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:09.785210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c35e73-081d-44dd-8867-1b2f70aeec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:09.851930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-47c35e73-081d-44dd-8867-1b2f70aeec19 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:09.853504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-47c35e73-081d-44dd-8867-1b2f70aeec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:10.062213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 852/1713] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:10.075895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 862/1714] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:10.240903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:10.403407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410b41-22e7-4aac-864f-5dcdef2f0797 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:10.404013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410b41-22e7-4aac-864f-5dcdef2f0797 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:10.404379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410b41-22e7-4aac-864f-5dcdef2f0797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:10.512463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4b410b41-22e7-4aac-864f-5dcdef2f0797 None None] MFA Rules not processed for user `2b27283f35d54b45b01b1da900eb2f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:10.513802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4b410b41-22e7-4aac-864f-5dcdef2f0797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:10.701298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 853/1715] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:10.702196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c35e73-081d-44dd-8867-1b2f70aeec19 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:10.703946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-47c35e73-081d-44dd-8867-1b2f70aeec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:10.723026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:10.766215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b410b41-22e7-4aac-864f-5dcdef2f0797 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:10.768358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4b410b41-22e7-4aac-864f-5dcdef2f0797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:10.852265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b410b41-22e7-4aac-864f-5dcdef2f0797 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:10.862505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97b849-6904-4004-a8b8-e870e66478b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:10.864521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b849-6904-4004-a8b8-e870e66478b5 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:10.864994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b849-6904-4004-a8b8-e870e66478b5 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:10.865417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b849-6904-4004-a8b8-e870e66478b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:10.889387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b849-6904-4004-a8b8-e870e66478b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:10.891625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b849-6904-4004-a8b8-e870e66478b5 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:10.892781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c35e73-081d-44dd-8867-1b2f70aeec19 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:10.907223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 854/1716] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:10.909338 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c01ab-89c0-4695-af7e-d4d58307b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:10.910213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c01ab-89c0-4695-af7e-d4d58307b2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:10.910751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c01ab-89c0-4695-af7e-d4d58307b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:10.911289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c01ab-89c0-4695-af7e-d4d58307b2a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:10.912136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c01ab-89c0-4695-af7e-d4d58307b2a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:10.916485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c01ab-89c0-4695-af7e-d4d58307b2a1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:11.118628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 863/1717] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:10 2022] POST /identity/v3/projects => generated 405 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:11.119984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97b849-6904-4004-a8b8-e870e66478b5 service nova] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:11.137673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97b849-6904-4004-a8b8-e870e66478b5 service nova] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:11.143120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c01ab-89c0-4695-af7e-d4d58307b2a1 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:11.273970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc02be-98fd-4a01-a54b-c5ef95e3ca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:11.275158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc02be-98fd-4a01-a54b-c5ef95e3ca0c admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:11.276769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc02be-98fd-4a01-a54b-c5ef95e3ca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:11.279727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc02be-98fd-4a01-a54b-c5ef95e3ca0c admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:11.283055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc02be-98fd-4a01-a54b-c5ef95e3ca0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:11.285613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc02be-98fd-4a01-a54b-c5ef95e3ca0c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:11.304685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:11.382665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c01ab-89c0-4695-af7e-d4d58307b2a1 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:11.393856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30d13f-e47b-4fb6-a945-09f7dfcf5127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:11.394944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30d13f-e47b-4fb6-a945-09f7dfcf5127 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:11.396130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30d13f-e47b-4fb6-a945-09f7dfcf5127 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:11.396130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30d13f-e47b-4fb6-a945-09f7dfcf5127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:11.402513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 855/1718] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:11 2022] POST /identity/v3/users => generated 309 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:11.406180 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc02be-98fd-4a01-a54b-c5ef95e3ca0c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:11.432794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30d13f-e47b-4fb6-a945-09f7dfcf5127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:11.434944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30d13f-e47b-4fb6-a945-09f7dfcf5127 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:11.449688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc02be-98fd-4a01-a54b-c5ef95e3ca0c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:11.449906 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 864/1719] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:11.472673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a93d31-1576-4b13-a24b-c307dc2c66fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:11.473353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a93d31-1576-4b13-a24b-c307dc2c66fa admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:11.475890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a93d31-1576-4b13-a24b-c307dc2c66fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:11.476381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a93d31-1576-4b13-a24b-c307dc2c66fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:11.478263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a93d31-1576-4b13-a24b-c307dc2c66fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:11.481451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a93d31-1576-4b13-a24b-c307dc2c66fa admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:11.522739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 856/1720] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:11.524760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30d13f-e47b-4fb6-a945-09f7dfcf5127 service nova] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:11.550716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30d13f-e47b-4fb6-a945-09f7dfcf5127 service nova] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:11.579654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e428c6-f5f2-4aa3-805e-bec9892b4fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:11.580386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e428c6-f5f2-4aa3-805e-bec9892b4fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:11.580667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e428c6-f5f2-4aa3-805e-bec9892b4fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:11.580938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e428c6-f5f2-4aa3-805e-bec9892b4fc2 admin admin] PATH_INFO: `/v3/projects/06e19a8cb96e42f99783049e8c506360/users/fde924e7463c41bfb819e9d881903596/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:11.633499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e428c6-f5f2-4aa3-805e-bec9892b4fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e19a8cb96e42f99783049e8c506360, user_id=fde924e7463c41bfb819e9d881903596, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:11.636481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e428c6-f5f2-4aa3-805e-bec9892b4fc2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:11.753239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 865/1721] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:11 2022] PUT /identity/v3/projects/06e19a8cb96e42f99783049e8c506360/users/fde924e7463c41bfb819e9d881903596/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:11.754095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a93d31-1576-4b13-a24b-c307dc2c66fa admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:11.780530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a93d31-1576-4b13-a24b-c307dc2c66fa admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:11.901442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75092467-932f-4167-a8d0-5ad3b7ef09a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:11.902743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75092467-932f-4167-a8d0-5ad3b7ef09a3 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:11.902743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75092467-932f-4167-a8d0-5ad3b7ef09a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:11.902743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75092467-932f-4167-a8d0-5ad3b7ef09a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:11.903427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75092467-932f-4167-a8d0-5ad3b7ef09a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:11.905186 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75092467-932f-4167-a8d0-5ad3b7ef09a3 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:11.915647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 857/1722] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:11.922302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e428c6-f5f2-4aa3-805e-bec9892b4fc2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:11.933314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e428c6-f5f2-4aa3-805e-bec9892b4fc2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:11.938908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0714e-fc19-4bf2-8b8b-b4bdbfd21508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:11.939634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0714e-fc19-4bf2-8b8b-b4bdbfd21508 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:11.939926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0714e-fc19-4bf2-8b8b-b4bdbfd21508 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:11.940199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0714e-fc19-4bf2-8b8b-b4bdbfd21508 admin admin] PATH_INFO: `/v3/projects/06e19a8cb96e42f99783049e8c506360/users/fde924e7463c41bfb819e9d881903596/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:11.942107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0714e-fc19-4bf2-8b8b-b4bdbfd21508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e19a8cb96e42f99783049e8c506360, user_id=fde924e7463c41bfb819e9d881903596, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:11.944899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0714e-fc19-4bf2-8b8b-b4bdbfd21508 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:11.965467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 866/1723] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:11 2022] PUT /identity/v3/projects/06e19a8cb96e42f99783049e8c506360/users/fde924e7463c41bfb819e9d881903596/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:13.406471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf57b3-e489-4583-b452-949feff90cdf None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:13.407069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf57b3-e489-4583-b452-949feff90cdf None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:13.407178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf57b3-e489-4583-b452-949feff90cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:13.447484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-82cf57b3-e489-4583-b452-949feff90cdf None None] MFA Rules not processed for user `fde924e7463c41bfb819e9d881903596`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:13.448877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-82cf57b3-e489-4583-b452-949feff90cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:13.534055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 858/1724] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:14.475613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1160f5-57f7-4877-b859-e4e3c9e040ac None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:14.475613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1160f5-57f7-4877-b859-e4e3c9e040ac None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:14.475613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1160f5-57f7-4877-b859-e4e3c9e040ac None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:14.475613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 867/1725] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 15:57:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:57:14.525941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17f37-ba5c-4ff6-836e-47845fd57ff5 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:14.526899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17f37-ba5c-4ff6-836e-47845fd57ff5 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:14.527430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17f37-ba5c-4ff6-836e-47845fd57ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:14.565366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-38b17f37-ba5c-4ff6-836e-47845fd57ff5 None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:14.567046 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-38b17f37-ba5c-4ff6-836e-47845fd57ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:14.797796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 859/1726] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 15:57:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:14.830492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122f624-28c1-40d0-9dd2-0d62c8f409c2 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:14.831126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122f624-28c1-40d0-9dd2-0d62c8f409c2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:14.831660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122f624-28c1-40d0-9dd2-0d62c8f409c2 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:14.833937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 868/1727] 10.222.0.94 () {54 vars in 961 bytes} [Fri Aug 26 15:57:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:57:14.843682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e515-f3be-47a8-a62c-88c5c2051b91 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:14.844116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e515-f3be-47a8-a62c-88c5c2051b91 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:14.844469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e515-f3be-47a8-a62c-88c5c2051b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:14.866055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:14.870218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5d59e515-f3be-47a8-a62c-88c5c2051b91 None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:14.871933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5d59e515-f3be-47a8-a62c-88c5c2051b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:14.885343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 860/1728] 10.222.0.94 () {58 vars in 1058 bytes} [Fri Aug 26 15:57:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:14.897009 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122f624-28c1-40d0-9dd2-0d62c8f409c2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:14.927203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122f624-28c1-40d0-9dd2-0d62c8f409c2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:14.936378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c922dd2-ba33-4c48-a24f-76d10b1e9ec9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:14.937590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922dd2-ba33-4c48-a24f-76d10b1e9ec9 service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:14.938088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922dd2-ba33-4c48-a24f-76d10b1e9ec9 service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:14.938772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c922dd2-ba33-4c48-a24f-76d10b1e9ec9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:14.961682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c922dd2-ba33-4c48-a24f-76d10b1e9ec9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:14.963544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c922dd2-ba33-4c48-a24f-76d10b1e9ec9 service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:14.975867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 869/1729] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:16.153059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020621a6-b916-47d6-b87c-826f3aa18b46 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:16.153658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020621a6-b916-47d6-b87c-826f3aa18b46 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:16.153658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020621a6-b916-47d6-b87c-826f3aa18b46 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:16.156067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 861/1730] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 15:57:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:57:16.161590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591619-f37b-42dc-9470-277d97183ff2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:16.165733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591619-f37b-42dc-9470-277d97183ff2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:16.165733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591619-f37b-42dc-9470-277d97183ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:16.194006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-44591619-f37b-42dc-9470-277d97183ff2 None None] MFA Rules not processed for user `94983527f4f044b683921034aa1829b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:16.195975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-44591619-f37b-42dc-9470-277d97183ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:16.213122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 870/1731] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:16.268225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020621a6-b916-47d6-b87c-826f3aa18b46 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:16.334457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020621a6-b916-47d6-b87c-826f3aa18b46 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:16.380725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfdc24-4e76-49ca-8be1-978b7aefc20f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:16.383195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfdc24-4e76-49ca-8be1-978b7aefc20f service placement] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:16.383478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfdc24-4e76-49ca-8be1-978b7aefc20f service placement] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:16.385333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfdc24-4e76-49ca-8be1-978b7aefc20f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:16.394468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44591619-f37b-42dc-9470-277d97183ff2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:16.425309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfdc24-4e76-49ca-8be1-978b7aefc20f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:16.428310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfdc24-4e76-49ca-8be1-978b7aefc20f service placement] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:16.445285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:16.471621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 862/1732] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:16.517777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44591619-f37b-42dc-9470-277d97183ff2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:16.523109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:16.524008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:16.524425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:16.524794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] PATH_INFO: `/v3/users/2b27283f35d54b45b01b1da900eb2f3a` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:16.526030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b27283f35d54b45b01b1da900eb2f3a)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:16.527235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: WARNING py.warnings [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 15:57:16.527235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: warnings.warn(msg) Aug 26 15:57:16.527235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: Aug 26 15:57:16.528597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:16.720059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b27283f35d54b45b01b1da900eb2f3a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.720823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b27283f35d54b45b01b1da900eb2f3a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.741199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b27283f35d54b45b01b1da900eb2f3a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.850178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b27283f35d54b45b01b1da900eb2f3a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.864409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b27283f35d54b45b01b1da900eb2f3a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:16.871346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 871/1733] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:16 2022] DELETE /identity/v3/users/2b27283f35d54b45b01b1da900eb2f3a => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:17.180208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfdc24-4e76-49ca-8be1-978b7aefc20f service placement] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:17.215638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfdc24-4e76-49ca-8be1-978b7aefc20f service placement] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:17.289541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:17.289541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:17.289541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:17.289541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] PATH_INFO: `/v3/projects/65bea261cb0c4ed39a65758ca8def241` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:17.290136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=65bea261cb0c4ed39a65758ca8def241)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:17.291090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:17.417245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65bea261cb0c4ed39a65758ca8def241', 'request_id': 'req-7d5cf366-c464-49e3-af4d-02f90edba6cf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.433225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65bea261cb0c4ed39a65758ca8def241', 'request_id': 'req-7d5cf366-c464-49e3-af4d-02f90edba6cf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.506670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65bea261cb0c4ed39a65758ca8def241', 'request_id': 'req-7d5cf366-c464-49e3-af4d-02f90edba6cf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.522690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] The token cache is being invalidate because project 65bea261cb0c4ed39a65758ca8def241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:17.523100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.523817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.525603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 863/1734] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:17 2022] DELETE /identity/v3/projects/65bea261cb0c4ed39a65758ca8def241 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:17.530022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:17.531610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:17.545244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93aab74-ad85-4d53-95d7-b0be2ccd6dbc admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:17.628011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:17.628840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:17.629458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:17.629458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] PATH_INFO: `/v3/users/fde924e7463c41bfb819e9d881903596` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:17.631070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde924e7463c41bfb819e9d881903596)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:17.636674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:17.801265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:17.828453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde924e7463c41bfb819e9d881903596'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.829328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde924e7463c41bfb819e9d881903596'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.854912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde924e7463c41bfb819e9d881903596'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.987247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde924e7463c41bfb819e9d881903596'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:17.995348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2c904c4b-32d2-42c4-85d3-27a7e87e6a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde924e7463c41bfb819e9d881903596'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.006883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 872/1735] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:17 2022] DELETE /identity/v3/users/fde924e7463c41bfb819e9d881903596 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:18.320015 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:18.321600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:18.337829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cf366-c464-49e3-af4d-02f90edba6cf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:18.415155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:18.415793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:18.416069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:18.416336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] PATH_INFO: `/v3/projects/06e19a8cb96e42f99783049e8c506360` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:18.417462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e19a8cb96e42f99783049e8c506360)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:18.419391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:18.437969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:18.524325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e19a8cb96e42f99783049e8c506360', 'request_id': 'req-aae21872-90a2-49a8-8921-5421df78d2ae'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.530210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e19a8cb96e42f99783049e8c506360', 'request_id': 'req-aae21872-90a2-49a8-8921-5421df78d2ae'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.609930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e19a8cb96e42f99783049e8c506360', 'request_id': 'req-aae21872-90a2-49a8-8921-5421df78d2ae'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.634213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] The token cache is being invalidate because project 06e19a8cb96e42f99783049e8c506360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:18.634511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.635429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-aae21872-90a2-49a8-8921-5421df78d2ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:18.638048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 864/1736] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:18 2022] DELETE /identity/v3/projects/06e19a8cb96e42f99783049e8c506360 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:18.643332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422caf-c3bc-4c3c-a21d-fe6daa4e7b71 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:18.648046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422caf-c3bc-4c3c-a21d-fe6daa4e7b71 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:18.648046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422caf-c3bc-4c3c-a21d-fe6daa4e7b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:18.694375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-19422caf-c3bc-4c3c-a21d-fe6daa4e7b71 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:18.697680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-19422caf-c3bc-4c3c-a21d-fe6daa4e7b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:18.762868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 873/1737] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:18.772234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785363d2-3d34-4c93-b74e-ea397ffe28bc None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:18.772547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785363d2-3d34-4c93-b74e-ea397ffe28bc None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:18.773243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785363d2-3d34-4c93-b74e-ea397ffe28bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:18.794419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-785363d2-3d34-4c93-b74e-ea397ffe28bc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:18.796313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-785363d2-3d34-4c93-b74e-ea397ffe28bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:18.805167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 865/1738] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:18.809255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19422caf-c3bc-4c3c-a21d-fe6daa4e7b71 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:18.820074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19422caf-c3bc-4c3c-a21d-fe6daa4e7b71 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:18.824879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:18.825519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:18.826004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:18.826523 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:18.827371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:18.828081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:18.829832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:18.843057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 874/1739] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:18.847555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785363d2-3d34-4c93-b74e-ea397ffe28bc None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:18.858579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785363d2-3d34-4c93-b74e-ea397ffe28bc None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:18.868063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c893f3-6bf7-4f8c-b2dd-3524615fe393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:18.868704 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c893f3-6bf7-4f8c-b2dd-3524615fe393 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:18.869568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c893f3-6bf7-4f8c-b2dd-3524615fe393 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:18.869568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c893f3-6bf7-4f8c-b2dd-3524615fe393 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:18.870153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c893f3-6bf7-4f8c-b2dd-3524615fe393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:18.875305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c893f3-6bf7-4f8c-b2dd-3524615fe393 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:18.984766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 866/1740] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:18 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:18.988495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.013895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ea523-c378-4e1c-9ef7-72cd929d3d6a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.100434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.101217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.101498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.101763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.102386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.104401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:19.210955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 875/1741] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:18 2022] POST /identity/v3/users => generated 285 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:19.219263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c893f3-6bf7-4f8c-b2dd-3524615fe393 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.250194 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c893f3-6bf7-4f8c-b2dd-3524615fe393 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.275105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba5714-3928-4f9b-9758-b11182f5de06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.276322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba5714-3928-4f9b-9758-b11182f5de06 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.276322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba5714-3928-4f9b-9758-b11182f5de06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.276322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba5714-3928-4f9b-9758-b11182f5de06 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.278598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba5714-3928-4f9b-9758-b11182f5de06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.283697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba5714-3928-4f9b-9758-b11182f5de06 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:19.287414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.289584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:19.300019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 867/1742] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:19.301344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba5714-3928-4f9b-9758-b11182f5de06 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.305023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986625e-a70d-4a7e-bde3-5a61cbf9481f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.310309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.312637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.312637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.312637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] PATH_INFO: `/v3/users/1810de94548541329acb0284420ca11a` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.312882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba5714-3928-4f9b-9758-b11182f5de06 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.312966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1810de94548541329acb0284420ca11a)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.314845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:19.317425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b89cc1-fe15-47fc-a887-f9e17cce99f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.318130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b89cc1-fe15-47fc-a887-f9e17cce99f6 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.318531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b89cc1-fe15-47fc-a887-f9e17cce99f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.318868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b89cc1-fe15-47fc-a887-f9e17cce99f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.319609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b89cc1-fe15-47fc-a887-f9e17cce99f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.321442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b89cc1-fe15-47fc-a887-f9e17cce99f6 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:19.335347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 868/1743] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:19.337915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b89cc1-fe15-47fc-a887-f9e17cce99f6 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.350511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b89cc1-fe15-47fc-a887-f9e17cce99f6 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.355802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06810b5-0ef4-48dc-ade6-dfa36bbedd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.356541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06810b5-0ef4-48dc-ade6-dfa36bbedd4b admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.356800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06810b5-0ef4-48dc-ade6-dfa36bbedd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.357837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06810b5-0ef4-48dc-ade6-dfa36bbedd4b admin admin] PATH_INFO: `/v3/projects/a4fbab127b72485599bdb61d3574643d/users/5d2ed99f9c1e44c8b717ec537c55bd18/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.373634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06810b5-0ef4-48dc-ade6-dfa36bbedd4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4fbab127b72485599bdb61d3574643d, user_id=5d2ed99f9c1e44c8b717ec537c55bd18, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.376641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06810b5-0ef4-48dc-ade6-dfa36bbedd4b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:19.478664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 869/1744] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:19 2022] PUT /identity/v3/projects/a4fbab127b72485599bdb61d3574643d/users/5d2ed99f9c1e44c8b717ec537c55bd18/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:19.492611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31854c30-3efd-427f-8a5e-3836bc493c9a None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.492825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:19.493387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31854c30-3efd-427f-8a5e-3836bc493c9a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.493694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31854c30-3efd-427f-8a5e-3836bc493c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.511005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1810de94548541329acb0284420ca11a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.511946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1810de94548541329acb0284420ca11a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.527744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-31854c30-3efd-427f-8a5e-3836bc493c9a None None] MFA Rules not processed for user `5d2ed99f9c1e44c8b717ec537c55bd18`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:19.531796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-31854c30-3efd-427f-8a5e-3836bc493c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:19.532094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1810de94548541329acb0284420ca11a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.593638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1810de94548541329acb0284420ca11a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.602729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-367ded93-61e0-49ec-b8d3-cace3328756d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1810de94548541329acb0284420ca11a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:19.608710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 870/1745] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:19.612794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 876/1746] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:19 2022] DELETE /identity/v3/users/1810de94548541329acb0284420ca11a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:19.616399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31854c30-3efd-427f-8a5e-3836bc493c9a None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.616685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d31d-9088-4037-9c7b-a6328cd207b4 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.616685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d31d-9088-4037-9c7b-a6328cd207b4 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.616685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05d31d-9088-4037-9c7b-a6328cd207b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.642426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-fa05d31d-9088-4037-9c7b-a6328cd207b4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:19.643974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:19.644712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-fa05d31d-9088-4037-9c7b-a6328cd207b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:19.729575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31854c30-3efd-427f-8a5e-3836bc493c9a None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.738353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 877/1747] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:19.748172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3afd1-2a07-4eab-882e-1fc7a0eba8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.748651 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05d31d-9088-4037-9c7b-a6328cd207b4 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.748973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3afd1-2a07-4eab-882e-1fc7a0eba8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.749249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3afd1-2a07-4eab-882e-1fc7a0eba8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.749523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3afd1-2a07-4eab-882e-1fc7a0eba8f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.750172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3afd1-2a07-4eab-882e-1fc7a0eba8f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.751326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-fa05d31d-9088-4037-9c7b-a6328cd207b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:19.753195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3afd1-2a07-4eab-882e-1fc7a0eba8f6 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:19.813650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05d31d-9088-4037-9c7b-a6328cd207b4 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.817621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.818275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.818718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.818984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.828905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.830527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:19.840672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 878/1748] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:19.866336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 871/1749] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:19 2022] POST /identity/v3/projects => generated 369 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:19.877841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:19.877841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:19.883303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b682ee-12e8-4440-a3de-cfc194e2b53b service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:19.967227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb3159-3787-40f0-b0d9-558125b5a317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:19.968038 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb3159-3787-40f0-b0d9-558125b5a317 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:19.968327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb3159-3787-40f0-b0d9-558125b5a317 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:19.968602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb3159-3787-40f0-b0d9-558125b5a317 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:19.969237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb3159-3787-40f0-b0d9-558125b5a317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:19.971341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb3159-3787-40f0-b0d9-558125b5a317 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.049473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 879/1750] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:19 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:20.053294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3afd1-2a07-4eab-882e-1fc7a0eba8f6 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.064929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3afd1-2a07-4eab-882e-1fc7a0eba8f6 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.070842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb885a0-d37f-4fbf-a7cd-e9674c7305bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.071488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb885a0-d37f-4fbf-a7cd-e9674c7305bb admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.073114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb885a0-d37f-4fbf-a7cd-e9674c7305bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.073114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb885a0-d37f-4fbf-a7cd-e9674c7305bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.073114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb885a0-d37f-4fbf-a7cd-e9674c7305bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.075214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb885a0-d37f-4fbf-a7cd-e9674c7305bb admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.101965 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 872/1751] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:20.106196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb3159-3787-40f0-b0d9-558125b5a317 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.118087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb3159-3787-40f0-b0d9-558125b5a317 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.125246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793494c-f8d1-49ab-a690-02270d234ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.125246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793494c-f8d1-49ab-a690-02270d234ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.125246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793494c-f8d1-49ab-a690-02270d234ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.125246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793494c-f8d1-49ab-a690-02270d234ce9 admin admin] PATH_INFO: `/v3/projects/be0b3b8d15d3452fb89316ef9f57b02f/users/c2cc87a05d3f4890bafb0c055097e3ee/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.143294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793494c-f8d1-49ab-a690-02270d234ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0b3b8d15d3452fb89316ef9f57b02f, user_id=c2cc87a05d3f4890bafb0c055097e3ee, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.146139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793494c-f8d1-49ab-a690-02270d234ce9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.229045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 880/1752] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:20 2022] PUT /identity/v3/projects/be0b3b8d15d3452fb89316ef9f57b02f/users/c2cc87a05d3f4890bafb0c055097e3ee/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:20.233570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb885a0-d37f-4fbf-a7cd-e9674c7305bb admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.248929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb885a0-d37f-4fbf-a7cd-e9674c7305bb admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.323400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05b9b5-6176-46aa-94fc-47a7512130f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.324058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b9b5-6176-46aa-94fc-47a7512130f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.324399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b9b5-6176-46aa-94fc-47a7512130f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.324893 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05b9b5-6176-46aa-94fc-47a7512130f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.325706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05b9b5-6176-46aa-94fc-47a7512130f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.331354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05b9b5-6176-46aa-94fc-47a7512130f4 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.346828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 873/1753] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:20.347869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793494c-f8d1-49ab-a690-02270d234ce9 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.365211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793494c-f8d1-49ab-a690-02270d234ce9 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.382157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.383163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.384071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.385021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] PATH_INFO: `/v3/projects/be0b3b8d15d3452fb89316ef9f57b02f/users/c2cc87a05d3f4890bafb0c055097e3ee/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.388156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0b3b8d15d3452fb89316ef9f57b02f, user_id=c2cc87a05d3f4890bafb0c055097e3ee, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.393097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.413319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 881/1754] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:20 2022] PUT /identity/v3/projects/be0b3b8d15d3452fb89316ef9f57b02f/users/c2cc87a05d3f4890bafb0c055097e3ee/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:20.425195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60051170-22e8-48a0-a278-b4476346c597 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.425482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60051170-22e8-48a0-a278-b4476346c597 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.425761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60051170-22e8-48a0-a278-b4476346c597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.469146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:20.477714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-60051170-22e8-48a0-a278-b4476346c597 None None] MFA Rules not processed for user `c2cc87a05d3f4890bafb0c055097e3ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:20.479047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-60051170-22e8-48a0-a278-b4476346c597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:20.562668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 874/1755] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:20.583005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.586049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:20.603985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60051170-22e8-48a0-a278-b4476346c597 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.605521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-60051170-22e8-48a0-a278-b4476346c597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:20.633640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:20.713023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d798b9-9b2f-45ef-9f6e-ee62c91760b0 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.722036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.727693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.727994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.728249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.730728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60051170-22e8-48a0-a278-b4476346c597 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.736445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:20.737090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:20.737513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:20.737932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] PATH_INFO: `/v3/projects/7279711958be4782a08f82bcfa02c2ab` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:20.739111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7279711958be4782a08f82bcfa02c2ab)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.739677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:20.741387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.741507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:20.754118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 882/1756] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:20.882805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7279711958be4782a08f82bcfa02c2ab', 'request_id': 'req-9684c0bf-913b-45d8-95dc-34c5a64e54ba'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:20.898511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7279711958be4782a08f82bcfa02c2ab', 'request_id': 'req-9684c0bf-913b-45d8-95dc-34c5a64e54ba'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:20.940943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:20.942382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:20.959143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56763d4e-fbf0-4cb3-be19-6ea1dfc16c18 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:20.976727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7279711958be4782a08f82bcfa02c2ab', 'request_id': 'req-9684c0bf-913b-45d8-95dc-34c5a64e54ba'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:20.994101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] The token cache is being invalidate because project 7279711958be4782a08f82bcfa02c2ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:20.994446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:20.995111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:20.998973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 875/1757] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:20 2022] DELETE /identity/v3/projects/7279711958be4782a08f82bcfa02c2ab => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:21.000992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:21.002625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:21.030551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.031220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.031502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.032000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.034053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:21.127483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684c0bf-913b-45d8-95dc-34c5a64e54ba admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:21.139570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:21.140229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:21.140504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:21.140826 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] PATH_INFO: `/v3/users/69c5bee0c3eb4f4896df20a0d74334ee` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:21.142265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c5bee0c3eb4f4896df20a0d74334ee)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.144266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.154866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:21.156935 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:21.171377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 883/1758] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:21.496318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c5bee0c3eb4f4896df20a0d74334ee'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:21.496974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c5bee0c3eb4f4896df20a0d74334ee'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:21.509362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c5bee0c3eb4f4896df20a0d74334ee'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:21.509999 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c5bee0c3eb4f4896df20a0d74334ee'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:21.558128 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:21.592796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8a6bd8ab-5d6a-405a-ba91-9f889e999609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c5bee0c3eb4f4896df20a0d74334ee'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:21.606832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 876/1759] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:21 2022] DELETE /identity/v3/users/69c5bee0c3eb4f4896df20a0d74334ee => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:22.623680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:22.632939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:22.641464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493baaf-9771-4278-8ae0-95b212ed601f service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:22.721215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:22.721951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:22.722664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:22.725708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] PATH_INFO: `/v3/projects/f053226c8ae94f3c8c602ac7a0187d15` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:22.727502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f053226c8ae94f3c8c602ac7a0187d15)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:22.732886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:22.851147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f053226c8ae94f3c8c602ac7a0187d15', 'request_id': 'req-1b5ab23b-fb84-4505-bf0c-af522fcb9590'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:22.856953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f053226c8ae94f3c8c602ac7a0187d15', 'request_id': 'req-1b5ab23b-fb84-4505-bf0c-af522fcb9590'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:22.913176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f053226c8ae94f3c8c602ac7a0187d15', 'request_id': 'req-1b5ab23b-fb84-4505-bf0c-af522fcb9590'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:22.928753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] The token cache is being invalidate because project f053226c8ae94f3c8c602ac7a0187d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:22.929043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:22.929747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-1b5ab23b-fb84-4505-bf0c-af522fcb9590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:22.931317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 884/1760] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:22 2022] DELETE /identity/v3/projects/f053226c8ae94f3c8c602ac7a0187d15 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:22.936230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158993b-4864-4e5f-b2d5-22cbdf2ae7a8 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:22.936687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158993b-4864-4e5f-b2d5-22cbdf2ae7a8 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:22.936970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158993b-4864-4e5f-b2d5-22cbdf2ae7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:22.957949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d158993b-4864-4e5f-b2d5-22cbdf2ae7a8 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:22.959308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d158993b-4864-4e5f-b2d5-22cbdf2ae7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:23.044286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 877/1761] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:23.054658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d925914-3cf9-48cf-b725-df7b7ece31fc None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:23.054658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d925914-3cf9-48cf-b725-df7b7ece31fc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:23.054658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d925914-3cf9-48cf-b725-df7b7ece31fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:23.083190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2d925914-3cf9-48cf-b725-df7b7ece31fc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:23.084938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2d925914-3cf9-48cf-b725-df7b7ece31fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:23.095426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 885/1762] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:23.100632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158993b-4864-4e5f-b2d5-22cbdf2ae7a8 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:23.117115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158993b-4864-4e5f-b2d5-22cbdf2ae7a8 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:23.128027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:23.129137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:23.129423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:23.129700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:23.130370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:23.130846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:23.140150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:23.156922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 878/1763] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:23.161385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d925914-3cf9-48cf-b725-df7b7ece31fc None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:23.171817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d925914-3cf9-48cf-b725-df7b7ece31fc None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:23.177430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88f318-2761-4fc0-8448-d72d98579ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:23.178276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88f318-2761-4fc0-8448-d72d98579ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:23.178551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88f318-2761-4fc0-8448-d72d98579ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:23.178963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88f318-2761-4fc0-8448-d72d98579ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:23.179693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88f318-2761-4fc0-8448-d72d98579ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:23.181658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88f318-2761-4fc0-8448-d72d98579ac5 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:23.314193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 886/1764] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:23 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:23.318328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:23.332566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3a1bf-7059-4a19-9222-403c881d9917 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:23.395124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b831709-c38e-4fc8-8e81-81b57693e49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:23.395737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b831709-c38e-4fc8-8e81-81b57693e49f admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:23.396018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b831709-c38e-4fc8-8e81-81b57693e49f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:23.396289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b831709-c38e-4fc8-8e81-81b57693e49f admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:23.396911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b831709-c38e-4fc8-8e81-81b57693e49f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:23.398827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b831709-c38e-4fc8-8e81-81b57693e49f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:23.496952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 879/1765] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:23 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:23.501977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88f318-2761-4fc0-8448-d72d98579ac5 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:23.523919 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88f318-2761-4fc0-8448-d72d98579ac5 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:23.533234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26e9d7-59bf-45ff-a271-530dfe4e9a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:23.533975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26e9d7-59bf-45ff-a271-530dfe4e9a41 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:23.534362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26e9d7-59bf-45ff-a271-530dfe4e9a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:23.534787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26e9d7-59bf-45ff-a271-530dfe4e9a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:23.535621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26e9d7-59bf-45ff-a271-530dfe4e9a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:23.540800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26e9d7-59bf-45ff-a271-530dfe4e9a41 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:23.580584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 887/1766] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:23.581774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b831709-c38e-4fc8-8e81-81b57693e49f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:23.598901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b831709-c38e-4fc8-8e81-81b57693e49f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:23.603943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf6f2-565a-486a-97f2-0f8d55839ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:23.604653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf6f2-565a-486a-97f2-0f8d55839ffd admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:23.606174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf6f2-565a-486a-97f2-0f8d55839ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:23.606174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf6f2-565a-486a-97f2-0f8d55839ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:23.606174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf6f2-565a-486a-97f2-0f8d55839ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:23.608086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf6f2-565a-486a-97f2-0f8d55839ffd admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:23.626144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26e9d7-59bf-45ff-a271-530dfe4e9a41 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:23.628244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 880/1767] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:23.639122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26e9d7-59bf-45ff-a271-530dfe4e9a41 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:23.644525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:23.645493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:23.646372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:23.646372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] PATH_INFO: `/v3/projects/a04a23a1c79144a7a9690ec900cf9de2/users/8654773473824a828f01c3ac22563ee3/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:23.662908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04a23a1c79144a7a9690ec900cf9de2, user_id=8654773473824a828f01c3ac22563ee3, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:23.665867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:23.764880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b2d40-65e1-4409-be51-2f54e02a3a89 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:23.764880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b2d40-65e1-4409-be51-2f54e02a3a89 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:23.764880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b2d40-65e1-4409-be51-2f54e02a3a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:23.785429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 888/1768] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:23 2022] PUT /identity/v3/projects/a04a23a1c79144a7a9690ec900cf9de2/users/8654773473824a828f01c3ac22563ee3/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:23.829694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-061b2d40-65e1-4409-be51-2f54e02a3a89 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:23.831667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-061b2d40-65e1-4409-be51-2f54e02a3a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:23.923009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:23.972981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 881/1769] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:23.983847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:23.986602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:24.032385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:24.069828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb43ca-e651-4389-a90d-24c61127c6f0 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:24.079555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9320f-9a84-4ae6-bb59-e1b3e318362f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:24.080202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9320f-9a84-4ae6-bb59-e1b3e318362f service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:24.080486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9320f-9a84-4ae6-bb59-e1b3e318362f service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:24.080755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9320f-9a84-4ae6-bb59-e1b3e318362f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:24.091059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9320f-9a84-4ae6-bb59-e1b3e318362f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:24.093798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9320f-9a84-4ae6-bb59-e1b3e318362f service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:24.104445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 889/1770] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:24.613318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd276-84fc-49dc-b5c9-30199a39465b None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:24.613692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd276-84fc-49dc-b5c9-30199a39465b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:24.618033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecd276-84fc-49dc-b5c9-30199a39465b None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:24.627037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 882/1771] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 15:57:24 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 15:57:24.715708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ea157-4b4d-46e4-9315-1c4f399c8b48 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:24.716171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ea157-4b4d-46e4-9315-1c4f399c8b48 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:24.716421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ea157-4b4d-46e4-9315-1c4f399c8b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:24.763347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2a3ea157-4b4d-46e4-9315-1c4f399c8b48 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:24.764939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2a3ea157-4b4d-46e4-9315-1c4f399c8b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:24.911366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 890/1772] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 15:57:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:24.940756 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecd276-84fc-49dc-b5c9-30199a39465b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:24.943064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d7ecd276-84fc-49dc-b5c9-30199a39465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:24.975668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecd276-84fc-49dc-b5c9-30199a39465b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:24.986193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b52b8-9ac5-468a-95ba-eff4aad1b3cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:24.986811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b52b8-9ac5-468a-95ba-eff4aad1b3cb service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:24.987231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b52b8-9ac5-468a-95ba-eff4aad1b3cb service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:24.987531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b52b8-9ac5-468a-95ba-eff4aad1b3cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:24.998580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b52b8-9ac5-468a-95ba-eff4aad1b3cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:25.000442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b52b8-9ac5-468a-95ba-eff4aad1b3cb service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:25.017762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 883/1773] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:25.725960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fb7e-0cfe-4dae-a941-29fb769f6500 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:25.726469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fb7e-0cfe-4dae-a941-29fb769f6500 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:25.726560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fb7e-0cfe-4dae-a941-29fb769f6500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:25.782776 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8511fb7e-0cfe-4dae-a941-29fb769f6500 None None] MFA Rules not processed for user `8654773473824a828f01c3ac22563ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:25.785247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8511fb7e-0cfe-4dae-a941-29fb769f6500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:25.984779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 891/1774] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:25.989506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b52b8-9ac5-468a-95ba-eff4aad1b3cb service nova] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:26.020912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b52b8-9ac5-468a-95ba-eff4aad1b3cb service nova] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:26.027436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd56d-acfa-4518-beca-934f22feb348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:26.028056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd56d-acfa-4518-beca-934f22feb348 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:26.028329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd56d-acfa-4518-beca-934f22feb348 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:26.028593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd56d-acfa-4518-beca-934f22feb348 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:26.029263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd56d-acfa-4518-beca-934f22feb348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:26.031204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd56d-acfa-4518-beca-934f22feb348 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:26.185870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 884/1775] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:26.189960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511fb7e-0cfe-4dae-a941-29fb769f6500 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:26.212354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511fb7e-0cfe-4dae-a941-29fb769f6500 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:26.281223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:26.290898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141358c-9f1a-409c-ad69-31369c3227c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:26.291803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141358c-9f1a-409c-ad69-31369c3227c5 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:26.292084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141358c-9f1a-409c-ad69-31369c3227c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:26.292729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141358c-9f1a-409c-ad69-31369c3227c5 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:26.292994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141358c-9f1a-409c-ad69-31369c3227c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:26.311949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141358c-9f1a-409c-ad69-31369c3227c5 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:26.441407 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 892/1776] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:26 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:26.442130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd56d-acfa-4518-beca-934f22feb348 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:26.471421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd56d-acfa-4518-beca-934f22feb348 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:26.491682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42678-fc85-4304-a20d-38088d59130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:26.492549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42678-fc85-4304-a20d-38088d59130f admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:26.493806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42678-fc85-4304-a20d-38088d59130f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:26.494296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f42678-fc85-4304-a20d-38088d59130f admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:26.496310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f42678-fc85-4304-a20d-38088d59130f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:26.502665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f42678-fc85-4304-a20d-38088d59130f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:26.523107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 885/1777] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:26.524757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141358c-9f1a-409c-ad69-31369c3227c5 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:26.538100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141358c-9f1a-409c-ad69-31369c3227c5 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:26.543998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233b308-8413-43a2-b353-a71cc3e2627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:26.545140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233b308-8413-43a2-b353-a71cc3e2627d admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:26.545582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233b308-8413-43a2-b353-a71cc3e2627d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:26.545963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233b308-8413-43a2-b353-a71cc3e2627d admin admin] PATH_INFO: `/v3/projects/8a93edea26904dda911fc965fdbd41b0/users/53e578d942d94258adebaa5b4b29d8f8/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:26.571051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233b308-8413-43a2-b353-a71cc3e2627d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a93edea26904dda911fc965fdbd41b0, user_id=53e578d942d94258adebaa5b4b29d8f8, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:26.574131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233b308-8413-43a2-b353-a71cc3e2627d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:26.722063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 893/1778] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:26 2022] PUT /identity/v3/projects/8a93edea26904dda911fc965fdbd41b0/users/53e578d942d94258adebaa5b4b29d8f8/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:26.725995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42678-fc85-4304-a20d-38088d59130f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:26.746075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f42678-fc85-4304-a20d-38088d59130f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:26.833637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:26.834270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:26.834553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:26.836234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:26.837074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:26.839036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:26.852630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 886/1779] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:26.859304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233b308-8413-43a2-b353-a71cc3e2627d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:26.872448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233b308-8413-43a2-b353-a71cc3e2627d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:26.878682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f88f6-eceb-4fdb-98e1-e24ddc328b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:26.879383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f88f6-eceb-4fdb-98e1-e24ddc328b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:26.879712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f88f6-eceb-4fdb-98e1-e24ddc328b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:26.879985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f88f6-eceb-4fdb-98e1-e24ddc328b69 admin admin] PATH_INFO: `/v3/projects/8a93edea26904dda911fc965fdbd41b0/users/53e578d942d94258adebaa5b4b29d8f8/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:26.882075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f88f6-eceb-4fdb-98e1-e24ddc328b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a93edea26904dda911fc965fdbd41b0, user_id=53e578d942d94258adebaa5b4b29d8f8, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:26.884893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f88f6-eceb-4fdb-98e1-e24ddc328b69 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:26.897335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 894/1780] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:26 2022] PUT /identity/v3/projects/8a93edea26904dda911fc965fdbd41b0/users/53e578d942d94258adebaa5b4b29d8f8/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:27.333545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:27.335278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:27.347844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8226c2d-5a57-4a79-93b8-b47816ae2e7c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:27.405771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:27.406352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:27.406669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:27.406999 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] PATH_INFO: `/v3/users/5d2ed99f9c1e44c8b717ec537c55bd18` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:27.408141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d2ed99f9c1e44c8b717ec537c55bd18)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:27.410073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:27.434448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:27.642889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d2ed99f9c1e44c8b717ec537c55bd18'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:27.643595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d2ed99f9c1e44c8b717ec537c55bd18'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:27.657544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d2ed99f9c1e44c8b717ec537c55bd18'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:27.658188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d2ed99f9c1e44c8b717ec537c55bd18'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:27.741698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d2ed99f9c1e44c8b717ec537c55bd18'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:27.755504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 887/1781] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:27 2022] DELETE /identity/v3/users/5d2ed99f9c1e44c8b717ec537c55bd18 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:27.760267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12e5c7-bd9b-44de-9b62-ce7df0ee31ac None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:27.760721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12e5c7-bd9b-44de-9b62-ce7df0ee31ac None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:27.761123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12e5c7-bd9b-44de-9b62-ce7df0ee31ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:27.783368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-9a12e5c7-bd9b-44de-9b62-ce7df0ee31ac None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:27.784603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9a12e5c7-bd9b-44de-9b62-ce7df0ee31ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:27.863545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 895/1782] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:27.874830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:27.876722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:27.952906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1824ad-0d93-4b13-ab64-623ac1d79f6f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:27.957329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3d473-9664-497f-ae42-1dd13462c351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:27.958397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3d473-9664-497f-ae42-1dd13462c351 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:27.958769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3d473-9664-497f-ae42-1dd13462c351 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:27.959324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3d473-9664-497f-ae42-1dd13462c351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:27.970263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3d473-9664-497f-ae42-1dd13462c351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:27.971744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3d473-9664-497f-ae42-1dd13462c351 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:27.982291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 888/1783] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:28.477908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12e5c7-bd9b-44de-9b62-ce7df0ee31ac None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:28.479366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9a12e5c7-bd9b-44de-9b62-ce7df0ee31ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:28.493365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12e5c7-bd9b-44de-9b62-ce7df0ee31ac None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:28.498714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:28.499412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:28.499693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:28.499964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] PATH_INFO: `/v3/projects/a4fbab127b72485599bdb61d3574643d` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:28.501233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4fbab127b72485599bdb61d3574643d)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:28.503579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:28.645667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4fbab127b72485599bdb61d3574643d', 'request_id': 'req-2f68df71-3373-401a-9f21-a0e26c6ce86f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.645952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a227a13-ae02-4f14-bb11-7b805846e3ce None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:28.646391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a227a13-ae02-4f14-bb11-7b805846e3ce None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:28.646391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a227a13-ae02-4f14-bb11-7b805846e3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:28.652559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4fbab127b72485599bdb61d3574643d', 'request_id': 'req-2f68df71-3373-401a-9f21-a0e26c6ce86f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.675400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4a227a13-ae02-4f14-bb11-7b805846e3ce None None] MFA Rules not processed for user `53e578d942d94258adebaa5b4b29d8f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:28.676635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4a227a13-ae02-4f14-bb11-7b805846e3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:28.753587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 889/1784] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:28.766132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a227a13-ae02-4f14-bb11-7b805846e3ce None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:28.767650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4a227a13-ae02-4f14-bb11-7b805846e3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:28.770434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4fbab127b72485599bdb61d3574643d', 'request_id': 'req-2f68df71-3373-401a-9f21-a0e26c6ce86f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.789262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] The token cache is being invalidate because project a4fbab127b72485599bdb61d3574643d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:28.789653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.790658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:28.792494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 896/1785] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:28 2022] DELETE /identity/v3/projects/a4fbab127b72485599bdb61d3574643d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:28.799897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:28.802116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:28.842288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a227a13-ae02-4f14-bb11-7b805846e3ce None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:28.846117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:28.846980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:28.847612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:28.847909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:28.848184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:28.850000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:28.886264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68df71-3373-401a-9f21-a0e26c6ce86f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:28.892551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:28.893442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:28.893884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:28.894151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] PATH_INFO: `/v3/users/c2cc87a05d3f4890bafb0c055097e3ee` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:28.895619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2cc87a05d3f4890bafb0c055097e3ee)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:28.897712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:28.921364 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:28.923085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:28.932790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 890/1786] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:29.097188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2cc87a05d3f4890bafb0c055097e3ee'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.097863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2cc87a05d3f4890bafb0c055097e3ee'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.110357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2cc87a05d3f4890bafb0c055097e3ee'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.151823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2cc87a05d3f4890bafb0c055097e3ee'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.159221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09a9eba2-a26e-41b0-853f-32fd6d9dfe6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2cc87a05d3f4890bafb0c055097e3ee'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.165390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 897/1787] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:28 2022] DELETE /identity/v3/users/c2cc87a05d3f4890bafb0c055097e3ee => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:29.657168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:29.668673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfe7ce-eed1-425e-8c8e-4a8a38a6e84a service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:29.730081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:29.730887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:29.731374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:29.732602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] PATH_INFO: `/v3/projects/be0b3b8d15d3452fb89316ef9f57b02f` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:29.734425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0b3b8d15d3452fb89316ef9f57b02f)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:29.736804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:29.827750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0b3b8d15d3452fb89316ef9f57b02f', 'request_id': 'req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.833093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0b3b8d15d3452fb89316ef9f57b02f', 'request_id': 'req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.901050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0b3b8d15d3452fb89316ef9f57b02f', 'request_id': 'req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.912900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] The token cache is being invalidate because project be0b3b8d15d3452fb89316ef9f57b02f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:29.913187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.913819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36d896f5-4290-4d8d-9cfd-eb1d8b7c8e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:29.915478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 891/1788] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:29 2022] DELETE /identity/v3/projects/be0b3b8d15d3452fb89316ef9f57b02f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:29.921109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e12c9-af07-4709-91f0-e6ef11cf3267 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:29.921567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e12c9-af07-4709-91f0-e6ef11cf3267 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:29.921652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e12c9-af07-4709-91f0-e6ef11cf3267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:29.942862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-478e12c9-af07-4709-91f0-e6ef11cf3267 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:29.944144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-478e12c9-af07-4709-91f0-e6ef11cf3267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:30.005410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 898/1789] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:30.013078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e90f-fb73-4a1d-89e5-9a469023efab None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.013361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e90f-fb73-4a1d-89e5-9a469023efab None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.013628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0e90f-fb73-4a1d-89e5-9a469023efab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.037570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-43b0e90f-fb73-4a1d-89e5-9a469023efab None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:30.038871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-43b0e90f-fb73-4a1d-89e5-9a469023efab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:30.049111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 892/1790] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:30.053643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e12c9-af07-4709-91f0-e6ef11cf3267 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:30.064543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e12c9-af07-4709-91f0-e6ef11cf3267 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:30.069136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:30.069740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.070012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.070280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.071040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:30.071337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:30.072966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:30.085664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 899/1791] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:57:30.090355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0e90f-fb73-4a1d-89e5-9a469023efab None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:30.100198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0e90f-fb73-4a1d-89e5-9a469023efab None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:30.105076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0fed-95a5-418f-b98c-dc0ae9e5cc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:30.105862 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0fed-95a5-418f-b98c-dc0ae9e5cc2d admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.106411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0fed-95a5-418f-b98c-dc0ae9e5cc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.106530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c0fed-95a5-418f-b98c-dc0ae9e5cc2d admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.107771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c0fed-95a5-418f-b98c-dc0ae9e5cc2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:30.109746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c0fed-95a5-418f-b98c-dc0ae9e5cc2d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:30.168125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:30.221876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 893/1792] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:30 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:30.226259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:30.237514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be018672-6a30-436c-950a-eaff4473501b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:30.293498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb99060-f9f4-42c2-b4c6-828213c49513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:30.294098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb99060-f9f4-42c2-b4c6-828213c49513 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.295020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb99060-f9f4-42c2-b4c6-828213c49513 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.295355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb99060-f9f4-42c2-b4c6-828213c49513 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.295986 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb99060-f9f4-42c2-b4c6-828213c49513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:30.297945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb99060-f9f4-42c2-b4c6-828213c49513 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:30.408303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 900/1793] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:30 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:30.414620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0fed-95a5-418f-b98c-dc0ae9e5cc2d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:30.426403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c0fed-95a5-418f-b98c-dc0ae9e5cc2d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:30.431077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c22232-b4f0-4fda-a3c2-ac6067b75132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:30.431683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22232-b4f0-4fda-a3c2-ac6067b75132 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.431961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22232-b4f0-4fda-a3c2-ac6067b75132 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.432334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c22232-b4f0-4fda-a3c2-ac6067b75132 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.432869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c22232-b4f0-4fda-a3c2-ac6067b75132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:30.434570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c22232-b4f0-4fda-a3c2-ac6067b75132 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:30.445283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 894/1794] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:30.449247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb99060-f9f4-42c2-b4c6-828213c49513 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:30.459892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb99060-f9f4-42c2-b4c6-828213c49513 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:30.465878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03cf8f-940b-49f4-b146-0e94efaac6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:30.467168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03cf8f-940b-49f4-b146-0e94efaac6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.467579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03cf8f-940b-49f4-b146-0e94efaac6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.467850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03cf8f-940b-49f4-b146-0e94efaac6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.468488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03cf8f-940b-49f4-b146-0e94efaac6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:30.471587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03cf8f-940b-49f4-b146-0e94efaac6a6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:30.487796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 901/1795] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:30.489173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c22232-b4f0-4fda-a3c2-ac6067b75132 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:30.501473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c22232-b4f0-4fda-a3c2-ac6067b75132 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:30.506521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b34a4-4faa-4577-af31-7427d31d4f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:30.507237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b34a4-4faa-4577-af31-7427d31d4f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.507518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b34a4-4faa-4577-af31-7427d31d4f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.507793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b34a4-4faa-4577-af31-7427d31d4f32 admin admin] PATH_INFO: `/v3/projects/b6f8e31d6c21459fa51a23a75c0aa2f1/users/834270abf57041508ed8e505896129b4/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.524813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b34a4-4faa-4577-af31-7427d31d4f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f8e31d6c21459fa51a23a75c0aa2f1, user_id=834270abf57041508ed8e505896129b4, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:30.527598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b34a4-4faa-4577-af31-7427d31d4f32 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:30.639277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 895/1796] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:30 2022] PUT /identity/v3/projects/b6f8e31d6c21459fa51a23a75c0aa2f1/users/834270abf57041508ed8e505896129b4/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:30.651198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5f0b-a955-4cb3-b381-a70863382951 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.651479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5f0b-a955-4cb3-b381-a70863382951 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.651745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5f0b-a955-4cb3-b381-a70863382951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.682615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8eaf5f0b-a955-4cb3-b381-a70863382951 None None] MFA Rules not processed for user `834270abf57041508ed8e505896129b4`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:30.683857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8eaf5f0b-a955-4cb3-b381-a70863382951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:30.756419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 902/1797] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:30.762390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b34a4-4faa-4577-af31-7427d31d4f32 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:30.781641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:30.841827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b34a4-4faa-4577-af31-7427d31d4f32 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:30.847482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1af73-d12a-4592-85b5-1c27cec493f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:30.848104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1af73-d12a-4592-85b5-1c27cec493f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:30.848372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1af73-d12a-4592-85b5-1c27cec493f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:30.848642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1af73-d12a-4592-85b5-1c27cec493f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:30.849270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1af73-d12a-4592-85b5-1c27cec493f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:30.851210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1af73-d12a-4592-85b5-1c27cec493f7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.018191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 896/1798] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:30 2022] POST /identity/v3/projects => generated 387 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:31.022333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf5f0b-a955-4cb3-b381-a70863382951 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.034342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf5f0b-a955-4cb3-b381-a70863382951 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.108720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2e387-a558-472e-9ed3-28405e470273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.109422 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2e387-a558-472e-9ed3-28405e470273 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.111014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2e387-a558-472e-9ed3-28405e470273 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.111125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2e387-a558-472e-9ed3-28405e470273 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.111736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2e387-a558-472e-9ed3-28405e470273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.113674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2e387-a558-472e-9ed3-28405e470273 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.194574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 903/1799] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:31 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:31.197605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1af73-d12a-4592-85b5-1c27cec493f7 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.210584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1af73-d12a-4592-85b5-1c27cec493f7 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.215273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865de3ca-018b-4071-a82a-d30fcd886865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.215882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865de3ca-018b-4071-a82a-d30fcd886865 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.216836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865de3ca-018b-4071-a82a-d30fcd886865 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.217272 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865de3ca-018b-4071-a82a-d30fcd886865 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.217916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865de3ca-018b-4071-a82a-d30fcd886865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.219524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865de3ca-018b-4071-a82a-d30fcd886865 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.230331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 897/1800] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:31.235558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2e387-a558-472e-9ed3-28405e470273 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.246545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2e387-a558-472e-9ed3-28405e470273 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.252247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c0aad-173e-444b-907b-3e5b6cb14fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.252247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0aad-173e-444b-907b-3e5b6cb14fea admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.252528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0aad-173e-444b-907b-3e5b6cb14fea admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.252905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0aad-173e-444b-907b-3e5b6cb14fea admin admin] PATH_INFO: `/v3/projects/1257c6b38a734b54896e60130ef27378/users/cbe02a1f00b04b878d9f947a8b6a6f5e/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.272710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c0aad-173e-444b-907b-3e5b6cb14fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1257c6b38a734b54896e60130ef27378, user_id=cbe02a1f00b04b878d9f947a8b6a6f5e, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.275839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c0aad-173e-444b-907b-3e5b6cb14fea admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.346469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 904/1801] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:31 2022] PUT /identity/v3/projects/1257c6b38a734b54896e60130ef27378/users/cbe02a1f00b04b878d9f947a8b6a6f5e/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:31.350628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865de3ca-018b-4071-a82a-d30fcd886865 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.360902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865de3ca-018b-4071-a82a-d30fcd886865 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.421851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d70c2-139b-4628-b12a-635472ef621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.422531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d70c2-139b-4628-b12a-635472ef621b admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.422812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d70c2-139b-4628-b12a-635472ef621b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.423107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d70c2-139b-4628-b12a-635472ef621b admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.423784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d70c2-139b-4628-b12a-635472ef621b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.425370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d70c2-139b-4628-b12a-635472ef621b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.438961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 898/1802] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:31.440462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c0aad-173e-444b-907b-3e5b6cb14fea admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.454142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c0aad-173e-444b-907b-3e5b6cb14fea admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.459402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.460068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.460343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.460608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] PATH_INFO: `/v3/projects/1257c6b38a734b54896e60130ef27378/users/cbe02a1f00b04b878d9f947a8b6a6f5e/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.463469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1257c6b38a734b54896e60130ef27378, user_id=cbe02a1f00b04b878d9f947a8b6a6f5e, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.466795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.480591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 905/1803] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:31 2022] PUT /identity/v3/projects/1257c6b38a734b54896e60130ef27378/users/cbe02a1f00b04b878d9f947a8b6a6f5e/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:31.490451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37fad6-5325-4fb7-87da-8fa1b74c2992 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.490676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37fad6-5325-4fb7-87da-8fa1b74c2992 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.490974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37fad6-5325-4fb7-87da-8fa1b74c2992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.520097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-df37fad6-5325-4fb7-87da-8fa1b74c2992 None None] MFA Rules not processed for user `cbe02a1f00b04b878d9f947a8b6a6f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:31.521325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-df37fad6-5325-4fb7-87da-8fa1b74c2992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:31.605517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:31.628643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 899/1804] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:31.643988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.646676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:31.709166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59d5fd-f922-4bef-be4e-625b53bb07bd admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.713270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774c7c9-8e9e-47bf-820b-5475bede836c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.713892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c7c9-8e9e-47bf-820b-5475bede836c service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.714166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c7c9-8e9e-47bf-820b-5475bede836c service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.714434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c7c9-8e9e-47bf-820b-5475bede836c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.724290 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774c7c9-8e9e-47bf-820b-5475bede836c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:31.725726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774c7c9-8e9e-47bf-820b-5475bede836c service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:31.735727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 906/1805] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:31.868873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37fad6-5325-4fb7-87da-8fa1b74c2992 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:31.870525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-df37fad6-5325-4fb7-87da-8fa1b74c2992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:31.886393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37fad6-5325-4fb7-87da-8fa1b74c2992 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:31.891249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392feb4-3410-42cd-919d-0b90d20fa25f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:31.891877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392feb4-3410-42cd-919d-0b90d20fa25f service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:31.892158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392feb4-3410-42cd-919d-0b90d20fa25f service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:31.892429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392feb4-3410-42cd-919d-0b90d20fa25f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:31.894609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e392feb4-3410-42cd-919d-0b90d20fa25f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:32.004334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392feb4-3410-42cd-919d-0b90d20fa25f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:32.005694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392feb4-3410-42cd-919d-0b90d20fa25f service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:32.017293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 900/1806] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:32.523203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:33.661347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774c7c9-8e9e-47bf-820b-5475bede836c service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:33.747517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774c7c9-8e9e-47bf-820b-5475bede836c service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:33.753000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:33.756987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:33.756987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:33.756987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] PATH_INFO: `/v3/users/834270abf57041508ed8e505896129b4` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:33.756987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834270abf57041508ed8e505896129b4)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:33.758573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:33.787636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:34.001945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834270abf57041508ed8e505896129b4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:34.002695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834270abf57041508ed8e505896129b4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:34.015428 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834270abf57041508ed8e505896129b4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:34.100790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834270abf57041508ed8e505896129b4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:34.110034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834270abf57041508ed8e505896129b4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:34.117669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 907/1807] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:33 2022] DELETE /identity/v3/users/834270abf57041508ed8e505896129b4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:34.123320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74b94-b5ba-47d2-a4eb-8f0338747d53 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.123581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74b94-b5ba-47d2-a4eb-8f0338747d53 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.123847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d74b94-b5ba-47d2-a4eb-8f0338747d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.143727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e6d74b94-b5ba-47d2-a4eb-8f0338747d53 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:34.144997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e6d74b94-b5ba-47d2-a4eb-8f0338747d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:34.204961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 901/1808] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:34.215218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.216707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:34.237225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:34.278023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ee567-4d55-4e22-9708-2ca5f32ea563 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.282002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea216e0-26fd-44cd-b679-d6e0a26ac89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.287445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea216e0-26fd-44cd-b679-d6e0a26ac89a service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.287445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea216e0-26fd-44cd-b679-d6e0a26ac89a service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.287445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea216e0-26fd-44cd-b679-d6e0a26ac89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.298976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea216e0-26fd-44cd-b679-d6e0a26ac89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.300470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea216e0-26fd-44cd-b679-d6e0a26ac89a service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.312965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 908/1809] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:34.792451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d74b94-b5ba-47d2-a4eb-8f0338747d53 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:34.794829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e6d74b94-b5ba-47d2-a4eb-8f0338747d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:34.809297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d74b94-b5ba-47d2-a4eb-8f0338747d53 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:34.815019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:34.815668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:34.816088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:34.816710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] PATH_INFO: `/v3/projects/b6f8e31d6c21459fa51a23a75c0aa2f1` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:34.817866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f8e31d6c21459fa51a23a75c0aa2f1)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:34.819895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:34.999074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:35.014853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f8e31d6c21459fa51a23a75c0aa2f1', 'request_id': 'req-671eaac0-2676-4d23-a919-53779ebdba80'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.022718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f8e31d6c21459fa51a23a75c0aa2f1', 'request_id': 'req-671eaac0-2676-4d23-a919-53779ebdba80'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.139565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f8e31d6c21459fa51a23a75c0aa2f1', 'request_id': 'req-671eaac0-2676-4d23-a919-53779ebdba80'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.175457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] The token cache is being invalidate because project b6f8e31d6c21459fa51a23a75c0aa2f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:35.175928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.178028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.183651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 902/1810] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:34 2022] DELETE /identity/v3/projects/b6f8e31d6c21459fa51a23a75c0aa2f1 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:35.188105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea216e0-26fd-44cd-b679-d6e0a26ac89a service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:35.203501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea216e0-26fd-44cd-b679-d6e0a26ac89a service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:35.288227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:35.289212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:35.289694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:35.290154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] PATH_INFO: `/v3/users/cbe02a1f00b04b878d9f947a8b6a6f5e` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:35.291781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe02a1f00b04b878d9f947a8b6a6f5e)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:35.294898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:35.494142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe02a1f00b04b878d9f947a8b6a6f5e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.494890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe02a1f00b04b878d9f947a8b6a6f5e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.511416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe02a1f00b04b878d9f947a8b6a6f5e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.587208 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe02a1f00b04b878d9f947a8b6a6f5e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.595472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-142ac693-29f6-4cda-9765-654634c10d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe02a1f00b04b878d9f947a8b6a6f5e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:35.601604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 909/1811] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:35 2022] DELETE /identity/v3/users/cbe02a1f00b04b878d9f947a8b6a6f5e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:36.071390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:36.072023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:36.099426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671eaac0-2676-4d23-a919-53779ebdba80 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:36.181548 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:36.182230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:36.182503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:36.182766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] PATH_INFO: `/v3/projects/1257c6b38a734b54896e60130ef27378` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:36.183916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1257c6b38a734b54896e60130ef27378)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:36.186453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:36.305465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1257c6b38a734b54896e60130ef27378', 'request_id': 'req-6bd149e6-552d-497e-96b3-d26f4e0d42dd'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:36.311379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1257c6b38a734b54896e60130ef27378', 'request_id': 'req-6bd149e6-552d-497e-96b3-d26f4e0d42dd'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:36.415467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1257c6b38a734b54896e60130ef27378', 'request_id': 'req-6bd149e6-552d-497e-96b3-d26f4e0d42dd'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:36.432170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] The token cache is being invalidate because project 1257c6b38a734b54896e60130ef27378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:36.432170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:36.432676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6bd149e6-552d-497e-96b3-d26f4e0d42dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:36.434675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 903/1812] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:36 2022] DELETE /identity/v3/projects/1257c6b38a734b54896e60130ef27378 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:36.439458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bbeaa-9b0c-4396-8008-a12701b90343 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:36.439929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bbeaa-9b0c-4396-8008-a12701b90343 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:36.440072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bbeaa-9b0c-4396-8008-a12701b90343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:36.462713 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d50bbeaa-9b0c-4396-8008-a12701b90343 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:36.464034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d50bbeaa-9b0c-4396-8008-a12701b90343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:36.517604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:36.549602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 910/1813] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:36.559039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803fca8-a76e-414e-8d4f-deabfaf3b7ee None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:36.559321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803fca8-a76e-414e-8d4f-deabfaf3b7ee None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:36.559598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803fca8-a76e-414e-8d4f-deabfaf3b7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:36.585211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c803fca8-a76e-414e-8d4f-deabfaf3b7ee None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:36.586473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c803fca8-a76e-414e-8d4f-deabfaf3b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:36.601434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 904/1814] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:36.606279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bbeaa-9b0c-4396-8008-a12701b90343 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:36.651272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bbeaa-9b0c-4396-8008-a12701b90343 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:36.651272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:36.651272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:36.651272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:36.652153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:36.652153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:36.652153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:36.652153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:36.678248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 911/1815] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:36.679460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803fca8-a76e-414e-8d4f-deabfaf3b7ee None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:36.694057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803fca8-a76e-414e-8d4f-deabfaf3b7ee None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:36.699534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bf087-173a-4935-9ba4-40bc26bbbe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:36.700251 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bf087-173a-4935-9ba4-40bc26bbbe3f admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:36.700784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bf087-173a-4935-9ba4-40bc26bbbe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:36.702689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bf087-173a-4935-9ba4-40bc26bbbe3f admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:36.702689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bf087-173a-4935-9ba4-40bc26bbbe3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:36.704227 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bf087-173a-4935-9ba4-40bc26bbbe3f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:36.829085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 905/1816] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:36 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:36.829961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:36.841985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5656b-bb03-4dfe-8deb-29d25e0bea70 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:36.949734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a348-dcce-4b81-959d-926f8d3203a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:36.951243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a348-dcce-4b81-959d-926f8d3203a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:36.951599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a348-dcce-4b81-959d-926f8d3203a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:36.951969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a348-dcce-4b81-959d-926f8d3203a8 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:36.952932 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2a348-dcce-4b81-959d-926f8d3203a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:36.955204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2a348-dcce-4b81-959d-926f8d3203a8 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:37.093332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 912/1817] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:36 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:37.099042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bf087-173a-4935-9ba4-40bc26bbbe3f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:37.114331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bf087-173a-4935-9ba4-40bc26bbbe3f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:37.120640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc01f5-a3d8-4141-a16b-4f9008df9e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:37.121371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc01f5-a3d8-4141-a16b-4f9008df9e76 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:37.121651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc01f5-a3d8-4141-a16b-4f9008df9e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:37.121962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc01f5-a3d8-4141-a16b-4f9008df9e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:37.123648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc01f5-a3d8-4141-a16b-4f9008df9e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:37.125946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc01f5-a3d8-4141-a16b-4f9008df9e76 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:37.148293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 906/1818] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:37.149229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a348-dcce-4b81-959d-926f8d3203a8 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:37.162599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a348-dcce-4b81-959d-926f8d3203a8 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:37.168632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c16df-af09-4da5-aeb4-cc56b04417e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:37.169065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c16df-af09-4da5-aeb4-cc56b04417e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:37.169509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c16df-af09-4da5-aeb4-cc56b04417e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:37.169795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c16df-af09-4da5-aeb4-cc56b04417e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:37.170527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c16df-af09-4da5-aeb4-cc56b04417e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:37.172173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c16df-af09-4da5-aeb4-cc56b04417e1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:37.185165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 913/1819] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:37.189361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc01f5-a3d8-4141-a16b-4f9008df9e76 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:37.202010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc01f5-a3d8-4141-a16b-4f9008df9e76 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:37.207196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:37.208032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:37.208457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:37.208725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] PATH_INFO: `/v3/projects/344787111544467a890e2c9bf4c2185f/users/284f5f828cbd4884b36f9595d514a470/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:37.219712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:37.228079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344787111544467a890e2c9bf4c2185f, user_id=284f5f828cbd4884b36f9595d514a470, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:37.232624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:37.391247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 907/1820] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:37 2022] PUT /identity/v3/projects/344787111544467a890e2c9bf4c2185f/users/284f5f828cbd4884b36f9595d514a470/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:37.401747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea07da4-b791-485f-87e2-7d691e7101bf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:37.402151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea07da4-b791-485f-87e2-7d691e7101bf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:37.402374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea07da4-b791-485f-87e2-7d691e7101bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:37.448230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5ea07da4-b791-485f-87e2-7d691e7101bf None None] MFA Rules not processed for user `284f5f828cbd4884b36f9595d514a470`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:37.450178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5ea07da4-b791-485f-87e2-7d691e7101bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:37.553143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 914/1821] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:37.858295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:37.860460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:37.969627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5c77-e919-4aa5-94ee-0e41d5636d65 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:37.976634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c469d68-583f-4aa6-a040-e37906fcf63b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:37.977785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c469d68-583f-4aa6-a040-e37906fcf63b service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:37.983236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c469d68-583f-4aa6-a040-e37906fcf63b service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:37.983236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c469d68-583f-4aa6-a040-e37906fcf63b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.001211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c469d68-583f-4aa6-a040-e37906fcf63b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:38.001211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c469d68-583f-4aa6-a040-e37906fcf63b service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:38.027177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 908/1822] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:38.067790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea07da4-b791-485f-87e2-7d691e7101bf None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:38.067790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5ea07da4-b791-485f-87e2-7d691e7101bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:38.129053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:38.190882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea07da4-b791-485f-87e2-7d691e7101bf None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:38.196250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:38.197004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.197353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.197691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] PATH_INFO: `/v3/users/17960d1d84f248fea5646a12bd6185e9` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.200043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17960d1d84f248fea5646a12bd6185e9)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:38.203358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:38.502705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17960d1d84f248fea5646a12bd6185e9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:38.504228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17960d1d84f248fea5646a12bd6185e9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:38.526364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17960d1d84f248fea5646a12bd6185e9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:38.618406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17960d1d84f248fea5646a12bd6185e9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:38.628703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17960d1d84f248fea5646a12bd6185e9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:38.637579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 915/1823] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:38 2022] DELETE /identity/v3/users/17960d1d84f248fea5646a12bd6185e9 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:38.644281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a1907-8705-4700-ae30-824e00bf224e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.644830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a1907-8705-4700-ae30-824e00bf224e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.644947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a1907-8705-4700-ae30-824e00bf224e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.667318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0f2a1907-8705-4700-ae30-824e00bf224e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:38.669022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0f2a1907-8705-4700-ae30-824e00bf224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:38.760135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 909/1824] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:38.766372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:38.767845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:38.847192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c8e5d-9029-4587-9310-8edf4bdbe4f9 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:38.850974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02440de6-6e42-465f-9a2a-08e86afbb82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:38.851683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02440de6-6e42-465f-9a2a-08e86afbb82c service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:38.851960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02440de6-6e42-465f-9a2a-08e86afbb82c service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:38.852230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02440de6-6e42-465f-9a2a-08e86afbb82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:38.869022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02440de6-6e42-465f-9a2a-08e86afbb82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:38.870705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02440de6-6e42-465f-9a2a-08e86afbb82c service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:38.882474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 916/1825] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:39.486914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a1907-8705-4700-ae30-824e00bf224e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:39.491827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0f2a1907-8705-4700-ae30-824e00bf224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:39.511658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a1907-8705-4700-ae30-824e00bf224e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:39.517988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:39.519188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:39.519649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:39.520100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] PATH_INFO: `/v3/projects/5acbcd6961414a07a2fc96da3da3ef0a` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:39.521528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5acbcd6961414a07a2fc96da3da3ef0a)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:39.524258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:39.656749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5acbcd6961414a07a2fc96da3da3ef0a', 'request_id': 'req-418764bd-eefc-49bc-af01-42a5e5a510c8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.662484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5acbcd6961414a07a2fc96da3da3ef0a', 'request_id': 'req-418764bd-eefc-49bc-af01-42a5e5a510c8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.750663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5acbcd6961414a07a2fc96da3da3ef0a', 'request_id': 'req-418764bd-eefc-49bc-af01-42a5e5a510c8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.763373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] The token cache is being invalidate because project 5acbcd6961414a07a2fc96da3da3ef0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:39.763661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.764321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:39.765893 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 910/1826] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:39 2022] DELETE /identity/v3/projects/5acbcd6961414a07a2fc96da3da3ef0a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:39.773145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02440de6-6e42-465f-9a2a-08e86afbb82c service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:39.782570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02440de6-6e42-465f-9a2a-08e86afbb82c service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:39.838239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:39.844173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:39.844799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:39.845070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:39.846531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] PATH_INFO: `/v3/users/93c7cf49da994f608e3baa51326928ed` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:39.846531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c7cf49da994f608e3baa51326928ed)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:39.848460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:40.226955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c7cf49da994f608e3baa51326928ed'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.227635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c7cf49da994f608e3baa51326928ed'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.251840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c7cf49da994f608e3baa51326928ed'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.380777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c7cf49da994f608e3baa51326928ed'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.394199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-96323a66-cdfe-4a45-9a55-a24fc5c9b8c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c7cf49da994f608e3baa51326928ed'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:40.402743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 917/1827] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:39 2022] DELETE /identity/v3/users/93c7cf49da994f608e3baa51326928ed => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:41.120160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:41.122222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:41.152476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418764bd-eefc-49bc-af01-42a5e5a510c8 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:41.166812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:41.219107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:41.219757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:41.222764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:41.222764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] PATH_INFO: `/v3/projects/4053e0b9a658474dad65780e015e84ea` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:41.222764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4053e0b9a658474dad65780e015e84ea)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:41.223393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:41.361582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4053e0b9a658474dad65780e015e84ea', 'request_id': 'req-598524aa-d9dc-4085-9d93-a2f75fd923fa'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.381829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4053e0b9a658474dad65780e015e84ea', 'request_id': 'req-598524aa-d9dc-4085-9d93-a2f75fd923fa'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.499732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4053e0b9a658474dad65780e015e84ea', 'request_id': 'req-598524aa-d9dc-4085-9d93-a2f75fd923fa'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.514503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] The token cache is being invalidate because project 4053e0b9a658474dad65780e015e84ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:41.514793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.515601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-598524aa-d9dc-4085-9d93-a2f75fd923fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:41.517355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 911/1828] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:41 2022] DELETE /identity/v3/projects/4053e0b9a658474dad65780e015e84ea => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:41.522284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6d7c3-f860-4b7b-88f3-64834cd0a3cc None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:41.523108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6d7c3-f860-4b7b-88f3-64834cd0a3cc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:41.523386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6d7c3-f860-4b7b-88f3-64834cd0a3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:41.545052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8fe6d7c3-f860-4b7b-88f3-64834cd0a3cc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:41.546289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8fe6d7c3-f860-4b7b-88f3-64834cd0a3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:41.616512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 918/1829] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:41.629611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633145d1-e78a-47a5-87f0-2bd0d4c41b50 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:41.629611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633145d1-e78a-47a5-87f0-2bd0d4c41b50 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:41.629611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633145d1-e78a-47a5-87f0-2bd0d4c41b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:41.647935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-633145d1-e78a-47a5-87f0-2bd0d4c41b50 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:41.650744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-633145d1-e78a-47a5-87f0-2bd0d4c41b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:41.661098 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 912/1830] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:41.665386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6d7c3-f860-4b7b-88f3-64834cd0a3cc None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:41.677541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6d7c3-f860-4b7b-88f3-64834cd0a3cc None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:41.682403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:41.682966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:41.683263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:41.683523 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:41.684180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:41.685015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:41.690179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:41.701861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 919/1831] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:41.708547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633145d1-e78a-47a5-87f0-2bd0d4c41b50 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:41.718941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633145d1-e78a-47a5-87f0-2bd0d4c41b50 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:41.725396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3be703-0ae5-4b50-ae8a-7e130f2dedc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:41.726007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be703-0ae5-4b50-ae8a-7e130f2dedc6 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:41.726275 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be703-0ae5-4b50-ae8a-7e130f2dedc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:41.726536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be703-0ae5-4b50-ae8a-7e130f2dedc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:41.727229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3be703-0ae5-4b50-ae8a-7e130f2dedc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:41.729133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3be703-0ae5-4b50-ae8a-7e130f2dedc6 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:41.837271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 913/1832] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:41 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:41.842020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:41.852727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba712066-427e-4d49-b6fd-1102281a8ff2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:41.934436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26dcef-b588-46f6-8ca3-7ca12bd2ee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:41.935139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26dcef-b588-46f6-8ca3-7ca12bd2ee89 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:41.935538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26dcef-b588-46f6-8ca3-7ca12bd2ee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:41.935809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26dcef-b588-46f6-8ca3-7ca12bd2ee89 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:41.936446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26dcef-b588-46f6-8ca3-7ca12bd2ee89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:41.940920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26dcef-b588-46f6-8ca3-7ca12bd2ee89 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:41.955379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:42.072792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 920/1833] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:41 2022] POST /identity/v3/users => generated 291 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:42.076981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3be703-0ae5-4b50-ae8a-7e130f2dedc6 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:42.088816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3be703-0ae5-4b50-ae8a-7e130f2dedc6 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:42.094787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bc490-f13d-475e-bd29-667a57ceb945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:42.095906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bc490-f13d-475e-bd29-667a57ceb945 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:42.096329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bc490-f13d-475e-bd29-667a57ceb945 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:42.096768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bc490-f13d-475e-bd29-667a57ceb945 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:42.097559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bc490-f13d-475e-bd29-667a57ceb945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:42.099354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bc490-f13d-475e-bd29-667a57ceb945 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:42.114697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 914/1834] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:42.121713 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26dcef-b588-46f6-8ca3-7ca12bd2ee89 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:42.136660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26dcef-b588-46f6-8ca3-7ca12bd2ee89 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:42.148815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5824bd-915f-47d1-aed5-5b8a8ae0889e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:42.149370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5824bd-915f-47d1-aed5-5b8a8ae0889e admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:42.149647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5824bd-915f-47d1-aed5-5b8a8ae0889e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:42.149916 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5824bd-915f-47d1-aed5-5b8a8ae0889e admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:42.150621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5824bd-915f-47d1-aed5-5b8a8ae0889e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:42.152321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5824bd-915f-47d1-aed5-5b8a8ae0889e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:42.175293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 921/1835] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:42.179698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bc490-f13d-475e-bd29-667a57ceb945 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:42.193895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bc490-f13d-475e-bd29-667a57ceb945 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:42.199119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf51db-feaf-476c-bdad-05e8175eaa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:42.199957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf51db-feaf-476c-bdad-05e8175eaa8d admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:42.200286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf51db-feaf-476c-bdad-05e8175eaa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:42.200646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf51db-feaf-476c-bdad-05e8175eaa8d admin admin] PATH_INFO: `/v3/projects/72a1a753450b4f409ab09fa258b3035e/users/5b29c5b523704340b47abcfe372a9177/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:42.223007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf51db-feaf-476c-bdad-05e8175eaa8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a1a753450b4f409ab09fa258b3035e, user_id=5b29c5b523704340b47abcfe372a9177, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:42.225961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf51db-feaf-476c-bdad-05e8175eaa8d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:42.331956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 915/1836] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:42 2022] PUT /identity/v3/projects/72a1a753450b4f409ab09fa258b3035e/users/5b29c5b523704340b47abcfe372a9177/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:42.343379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac880c4c-77cc-48d1-9e7f-ce33035668c3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:42.344844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac880c4c-77cc-48d1-9e7f-ce33035668c3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:42.344844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac880c4c-77cc-48d1-9e7f-ce33035668c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:42.381157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ac880c4c-77cc-48d1-9e7f-ce33035668c3 None None] MFA Rules not processed for user `5b29c5b523704340b47abcfe372a9177`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:42.382949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ac880c4c-77cc-48d1-9e7f-ce33035668c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:42.487071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 922/1837] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:42.490528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf51db-feaf-476c-bdad-05e8175eaa8d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:42.545647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:42.582528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf51db-feaf-476c-bdad-05e8175eaa8d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:42.588758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de166d-d1a8-44d8-987e-a2b77dd6864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:42.589865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de166d-d1a8-44d8-987e-a2b77dd6864f admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:42.589865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de166d-d1a8-44d8-987e-a2b77dd6864f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:42.590089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de166d-d1a8-44d8-987e-a2b77dd6864f admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:42.590641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de166d-d1a8-44d8-987e-a2b77dd6864f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:42.592835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de166d-d1a8-44d8-987e-a2b77dd6864f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:42.702654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 916/1838] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:42 2022] POST /identity/v3/projects => generated 381 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:42.706726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac880c4c-77cc-48d1-9e7f-ce33035668c3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:42.718204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac880c4c-77cc-48d1-9e7f-ce33035668c3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:42.779692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dc26f-17fa-49ce-828f-e448a769f43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:42.781057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dc26f-17fa-49ce-828f-e448a769f43a admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:42.781378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dc26f-17fa-49ce-828f-e448a769f43a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:42.782179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dc26f-17fa-49ce-828f-e448a769f43a admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:42.782292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8dc26f-17fa-49ce-828f-e448a769f43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:42.784287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8dc26f-17fa-49ce-828f-e448a769f43a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:42.903364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 923/1839] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:42 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:42.907539 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de166d-d1a8-44d8-987e-a2b77dd6864f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:42.920351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de166d-d1a8-44d8-987e-a2b77dd6864f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:42.927010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348c5b-64ae-4d21-9037-2fb03ef4c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:42.927720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348c5b-64ae-4d21-9037-2fb03ef4c595 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:42.928096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348c5b-64ae-4d21-9037-2fb03ef4c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:42.928460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348c5b-64ae-4d21-9037-2fb03ef4c595 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:42.929214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7348c5b-64ae-4d21-9037-2fb03ef4c595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:42.931395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7348c5b-64ae-4d21-9037-2fb03ef4c595 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:42.944503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 917/1840] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:42.948546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dc26f-17fa-49ce-828f-e448a769f43a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:42.960129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dc26f-17fa-49ce-828f-e448a769f43a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:42.965167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71da774-a8b5-4157-90ab-e06f3731a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:42.965503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71da774-a8b5-4157-90ab-e06f3731a67e admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:42.966408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71da774-a8b5-4157-90ab-e06f3731a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:42.966846 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71da774-a8b5-4157-90ab-e06f3731a67e admin admin] PATH_INFO: `/v3/projects/6f71465f0e6741689dc4860c243d7a23/users/de278cfffe334fd08ae5ebd287bd15f1/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:42.984592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71da774-a8b5-4157-90ab-e06f3731a67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f71465f0e6741689dc4860c243d7a23, user_id=de278cfffe334fd08ae5ebd287bd15f1, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:42.987320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71da774-a8b5-4157-90ab-e06f3731a67e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:43.104969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 924/1841] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:42 2022] PUT /identity/v3/projects/6f71465f0e6741689dc4860c243d7a23/users/de278cfffe334fd08ae5ebd287bd15f1/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:43.106227 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348c5b-64ae-4d21-9037-2fb03ef4c595 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:43.118104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348c5b-64ae-4d21-9037-2fb03ef4c595 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:43.186088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12579d3-f2b1-4589-bca9-f9cc42105792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:43.186746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12579d3-f2b1-4589-bca9-f9cc42105792 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:43.187000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12579d3-f2b1-4589-bca9-f9cc42105792 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:43.187276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12579d3-f2b1-4589-bca9-f9cc42105792 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:43.187970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12579d3-f2b1-4589-bca9-f9cc42105792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:43.189939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12579d3-f2b1-4589-bca9-f9cc42105792 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:43.204842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 918/1842] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:43.206082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71da774-a8b5-4157-90ab-e06f3731a67e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:43.221189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71da774-a8b5-4157-90ab-e06f3731a67e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:43.238746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:43.239537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:43.240369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:43.240479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] PATH_INFO: `/v3/projects/6f71465f0e6741689dc4860c243d7a23/users/de278cfffe334fd08ae5ebd287bd15f1/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:43.242720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f71465f0e6741689dc4860c243d7a23, user_id=de278cfffe334fd08ae5ebd287bd15f1, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:43.245759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:43.259926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 925/1843] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:43 2022] PUT /identity/v3/projects/6f71465f0e6741689dc4860c243d7a23/users/de278cfffe334fd08ae5ebd287bd15f1/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:43.271363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7905e7-7690-4556-bcbc-b82cdb8c08b6 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:43.271651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7905e7-7690-4556-bcbc-b82cdb8c08b6 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:43.271930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7905e7-7690-4556-bcbc-b82cdb8c08b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:43.307616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-2a7905e7-7690-4556-bcbc-b82cdb8c08b6 None None] MFA Rules not processed for user `de278cfffe334fd08ae5ebd287bd15f1`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:43.308855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2a7905e7-7690-4556-bcbc-b82cdb8c08b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:43.410491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 919/1844] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:43.419947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:43.422122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:43.468523 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:43.514311 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c1a17-5912-4520-a09a-03f4f13f53f4 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:43.518146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2d51-7a68-465f-960c-583128b4133d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:43.518758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2d51-7a68-465f-960c-583128b4133d service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:43.519070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2d51-7a68-465f-960c-583128b4133d service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:43.519343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f2d51-7a68-465f-960c-583128b4133d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:43.529734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f2d51-7a68-465f-960c-583128b4133d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:43.532339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f2d51-7a68-465f-960c-583128b4133d service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:43.544570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 926/1845] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:44.625884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7905e7-7690-4556-bcbc-b82cdb8c08b6 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:44.635938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7905e7-7690-4556-bcbc-b82cdb8c08b6 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:44.640941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de278cfffe334fd08ae5ebd287bd15f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f71465f0e6741689dc4860c243d7a23', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:44.641555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:44.641829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:44.642096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:44.642910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:44.643238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:44.644841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:44.662991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 920/1846] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:44.663767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2d51-7a68-465f-960c-583128b4133d service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:44.676804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f2d51-7a68-465f-960c-583128b4133d service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:44.681448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b4653-89a9-4b19-9c36-8260d9d301eb tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de278cfffe334fd08ae5ebd287bd15f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f71465f0e6741689dc4860c243d7a23', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:44.682866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b4653-89a9-4b19-9c36-8260d9d301eb tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:44.683406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b4653-89a9-4b19-9c36-8260d9d301eb tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:44.683781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b4653-89a9-4b19-9c36-8260d9d301eb tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:44.684570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b4653-89a9-4b19-9c36-8260d9d301eb tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:44.686800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b4653-89a9-4b19-9c36-8260d9d301eb tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:44.797612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 927/1847] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:44 2022] POST /identity/v3/projects => generated 354 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:44.798739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:44.808888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b5941-94ef-4739-80e0-8f36485ef703 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:44.829955 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:44.892796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de278cfffe334fd08ae5ebd287bd15f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f71465f0e6741689dc4860c243d7a23', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:44.894136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:44.894477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:44.896169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:44.896169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:44.896169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:44.897897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:44.911705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 921/1848] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 15:57:44.960760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b4653-89a9-4b19-9c36-8260d9d301eb tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:44.972518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b4653-89a9-4b19-9c36-8260d9d301eb tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:44.978209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de278cfffe334fd08ae5ebd287bd15f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f71465f0e6741689dc4860c243d7a23', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:44.978943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:44.979244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:44.979501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] PATH_INFO: `/v3/projects/943e5f0b8e90421c8c62715df7c71e5d` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:44.980703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Authorizing `identity:delete_project(project_id=943e5f0b8e90421c8c62715df7c71e5d)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:44.983676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:45.101934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943e5f0b8e90421c8c62715df7c71e5d', 'request_id': 'req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:45.110039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943e5f0b8e90421c8c62715df7c71e5d', 'request_id': 'req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:45.209167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943e5f0b8e90421c8c62715df7c71e5d', 'request_id': 'req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:45.227385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] The token cache is being invalidate because project 943e5f0b8e90421c8c62715df7c71e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:45.227751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:45.228500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-163be5a2-2f10-47ae-ae06-6e4bdf943b9e tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:45.230974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 928/1849] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:44 2022] DELETE /identity/v3/projects/943e5f0b8e90421c8c62715df7c71e5d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:46.989894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:46.992391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:47.064850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966612a5-d529-4f16-9ae2-7a2a3f0766e9 tempest-VolumeQuotaClassesTest-1385522751 tempest-VolumeQuotaClassesTest-1385522751-project-admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:47.129395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:47.130046 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:47.130321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:47.130591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] PATH_INFO: `/v3/users/5b29c5b523704340b47abcfe372a9177` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:47.132422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b29c5b523704340b47abcfe372a9177)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:47.133788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:47.434166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b29c5b523704340b47abcfe372a9177'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:47.435361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b29c5b523704340b47abcfe372a9177'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:47.456980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b29c5b523704340b47abcfe372a9177'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:47.456980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b29c5b523704340b47abcfe372a9177'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:47.456980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:47.579977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b29c5b523704340b47abcfe372a9177'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:47.593333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 922/1850] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:46 2022] DELETE /identity/v3/users/5b29c5b523704340b47abcfe372a9177 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:47.603262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2beb5-ce44-4f49-897c-4a749b1d40c9 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:47.603740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2beb5-ce44-4f49-897c-4a749b1d40c9 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:47.603852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2beb5-ce44-4f49-897c-4a749b1d40c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:47.624246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3ee2beb5-ce44-4f49-897c-4a749b1d40c9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:47.628964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3ee2beb5-ce44-4f49-897c-4a749b1d40c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:47.687332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 929/1851] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:47.697100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:47.698492 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:47.761636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fe6e2-4013-413f-8da3-edcd607bad77 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:47.765655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233826-001d-40fd-983d-5895ad816d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:47.766322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233826-001d-40fd-983d-5895ad816d59 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:47.766617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233826-001d-40fd-983d-5895ad816d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:47.766891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233826-001d-40fd-983d-5895ad816d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:47.776139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28233826-001d-40fd-983d-5895ad816d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:47.777634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28233826-001d-40fd-983d-5895ad816d59 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:47.788077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 923/1852] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:48.284242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2beb5-ce44-4f49-897c-4a749b1d40c9 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:48.286026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3ee2beb5-ce44-4f49-897c-4a749b1d40c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:48.304105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2beb5-ce44-4f49-897c-4a749b1d40c9 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:48.313561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:48.313561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:48.313561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:48.314167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] PATH_INFO: `/v3/projects/72a1a753450b4f409ab09fa258b3035e` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:48.314828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a1a753450b4f409ab09fa258b3035e)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:48.316903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:48.347871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:48.413668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a1a753450b4f409ab09fa258b3035e', 'request_id': 'req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.420101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a1a753450b4f409ab09fa258b3035e', 'request_id': 'req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.477816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a1a753450b4f409ab09fa258b3035e', 'request_id': 'req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.490964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] The token cache is being invalidate because project 72a1a753450b4f409ab09fa258b3035e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:48.491232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.491858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.493355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 930/1853] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:48 2022] DELETE /identity/v3/projects/72a1a753450b4f409ab09fa258b3035e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:48.498269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233826-001d-40fd-983d-5895ad816d59 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:48.508299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233826-001d-40fd-983d-5895ad816d59 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:48.571695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:48.572415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:48.572758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:48.573100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] PATH_INFO: `/v3/users/de278cfffe334fd08ae5ebd287bd15f1` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:48.574302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=de278cfffe334fd08ae5ebd287bd15f1)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:48.576819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:48.875275 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de278cfffe334fd08ae5ebd287bd15f1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.875932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de278cfffe334fd08ae5ebd287bd15f1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.888166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de278cfffe334fd08ae5ebd287bd15f1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.888811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de278cfffe334fd08ae5ebd287bd15f1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.961558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d7b39e6-e1c6-409f-b87f-e39d5ddf73ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de278cfffe334fd08ae5ebd287bd15f1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:48.975671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 924/1854] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:48 2022] DELETE /identity/v3/users/de278cfffe334fd08ae5ebd287bd15f1 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:49.387939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:49.390228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:49.407363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f509ecc-62b3-409b-b2a4-4e6e95fc473d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:49.476742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:49.477400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:49.477681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:49.477956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] PATH_INFO: `/v3/projects/6f71465f0e6741689dc4860c243d7a23` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:49.479325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f71465f0e6741689dc4860c243d7a23)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:49.481816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:49.631782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f71465f0e6741689dc4860c243d7a23', 'request_id': 'req-cddf29d4-7b21-4ec1-b3d2-13582d404329'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:49.638689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f71465f0e6741689dc4860c243d7a23', 'request_id': 'req-cddf29d4-7b21-4ec1-b3d2-13582d404329'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:49.774821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f71465f0e6741689dc4860c243d7a23', 'request_id': 'req-cddf29d4-7b21-4ec1-b3d2-13582d404329'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:49.787780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] The token cache is being invalidate because project 6f71465f0e6741689dc4860c243d7a23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:49.788074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:49.788758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cddf29d4-7b21-4ec1-b3d2-13582d404329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:49.790294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 931/1855] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:49 2022] DELETE /identity/v3/projects/6f71465f0e6741689dc4860c243d7a23 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:49.795300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc49213-7c57-4029-907c-20a2992b997f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:49.795870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc49213-7c57-4029-907c-20a2992b997f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:49.795972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc49213-7c57-4029-907c-20a2992b997f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:49.804512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:49.815866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-fcc49213-7c57-4029-907c-20a2992b997f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:49.817084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-fcc49213-7c57-4029-907c-20a2992b997f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:49.877110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 925/1856] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:49.884745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff6e2-f8dd-499a-a4e4-f31245ee3f6f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:49.885031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff6e2-f8dd-499a-a4e4-f31245ee3f6f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:49.885303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff6e2-f8dd-499a-a4e4-f31245ee3f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:49.906926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b44ff6e2-f8dd-499a-a4e4-f31245ee3f6f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:49.908135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b44ff6e2-f8dd-499a-a4e4-f31245ee3f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:49.916019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 932/1857] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:49.920195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc49213-7c57-4029-907c-20a2992b997f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:49.930049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc49213-7c57-4029-907c-20a2992b997f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:49.935493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:49.936123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:49.936422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:49.936690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:49.937334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:49.937631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:49.939478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:49.951888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 926/1858] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:49.956587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff6e2-f8dd-499a-a4e4-f31245ee3f6f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:49.965974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff6e2-f8dd-499a-a4e4-f31245ee3f6f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:49.970669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5fe2-9d9e-489d-a7d4-6fe6dd656636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:49.971404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5fe2-9d9e-489d-a7d4-6fe6dd656636 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:49.977876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5fe2-9d9e-489d-a7d4-6fe6dd656636 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:49.977876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5fe2-9d9e-489d-a7d4-6fe6dd656636 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:49.977876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c5fe2-9d9e-489d-a7d4-6fe6dd656636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:49.977876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c5fe2-9d9e-489d-a7d4-6fe6dd656636 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:50.068621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 933/1859] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:49 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:50.072812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:50.083652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86788cef-a670-4017-8d05-e2e768f202db admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:50.140067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84526ed0-dcdf-41b8-b264-1a706ba3c585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:50.140686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84526ed0-dcdf-41b8-b264-1a706ba3c585 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:50.140966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84526ed0-dcdf-41b8-b264-1a706ba3c585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:50.141239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84526ed0-dcdf-41b8-b264-1a706ba3c585 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:50.141876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84526ed0-dcdf-41b8-b264-1a706ba3c585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:50.143852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84526ed0-dcdf-41b8-b264-1a706ba3c585 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:50.263095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 927/1860] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:50 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:50.264349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5fe2-9d9e-489d-a7d4-6fe6dd656636 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:50.277420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5fe2-9d9e-489d-a7d4-6fe6dd656636 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:50.282272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b8e6-c47c-4263-9269-54d992b377b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:50.282878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b8e6-c47c-4263-9269-54d992b377b6 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:50.283187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b8e6-c47c-4263-9269-54d992b377b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:50.283456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b8e6-c47c-4263-9269-54d992b377b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:50.284089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317b8e6-c47c-4263-9269-54d992b377b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:50.285694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317b8e6-c47c-4263-9269-54d992b377b6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:50.297050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 934/1861] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:50.316543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84526ed0-dcdf-41b8-b264-1a706ba3c585 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:50.326817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84526ed0-dcdf-41b8-b264-1a706ba3c585 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:50.331915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e565004-a9f5-4751-bf46-596a79982e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:50.332519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e565004-a9f5-4751-bf46-596a79982e5f admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:50.332800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e565004-a9f5-4751-bf46-596a79982e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:50.333068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e565004-a9f5-4751-bf46-596a79982e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:50.333703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e565004-a9f5-4751-bf46-596a79982e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:50.335389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e565004-a9f5-4751-bf46-596a79982e5f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:50.345974 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 928/1862] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:50.350084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b8e6-c47c-4263-9269-54d992b377b6 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:50.355397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:50.361749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b8e6-c47c-4263-9269-54d992b377b6 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:50.366300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:50.367020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:50.367303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:50.367582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] PATH_INFO: `/v3/projects/1841f10ca42740c3a3b69fed3e15ec51/users/32fcbaefaa6542639d840c982d7366b0/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:50.382970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1841f10ca42740c3a3b69fed3e15ec51, user_id=32fcbaefaa6542639d840c982d7366b0, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:50.386786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:50.482621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 935/1863] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:50 2022] PUT /identity/v3/projects/1841f10ca42740c3a3b69fed3e15ec51/users/32fcbaefaa6542639d840c982d7366b0/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:50.496904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d14fc-e68c-452a-9caa-928a614fdacd None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:50.497282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d14fc-e68c-452a-9caa-928a614fdacd None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:50.497687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d14fc-e68c-452a-9caa-928a614fdacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:50.526105 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-880d14fc-e68c-452a-9caa-928a614fdacd None None] MFA Rules not processed for user `32fcbaefaa6542639d840c982d7366b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:50.527367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-880d14fc-e68c-452a-9caa-928a614fdacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:50.592954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 929/1864] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:50.603607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:50.605262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:50.615866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:50.666728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0938f-94f5-4687-b39c-2a296bc9134e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:50.671661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536862ea-15a5-4ea4-8d26-4ce18a007456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:50.672379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536862ea-15a5-4ea4-8d26-4ce18a007456 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:50.672711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536862ea-15a5-4ea4-8d26-4ce18a007456 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:50.673046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536862ea-15a5-4ea4-8d26-4ce18a007456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:50.685113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536862ea-15a5-4ea4-8d26-4ce18a007456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:50.686978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536862ea-15a5-4ea4-8d26-4ce18a007456 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:50.697475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 936/1865] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:50.886209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d14fc-e68c-452a-9caa-928a614fdacd None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:50.888399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-880d14fc-e68c-452a-9caa-928a614fdacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:50.957345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d14fc-e68c-452a-9caa-928a614fdacd None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:50.961984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:50.962628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:50.962904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:50.963229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] PATH_INFO: `/v3/users/284f5f828cbd4884b36f9595d514a470` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:50.964350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] RBAC: Authorizing `identity:delete_user(user_id=284f5f828cbd4884b36f9595d514a470)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:50.967320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:51.149259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284f5f828cbd4884b36f9595d514a470'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:51.149558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284f5f828cbd4884b36f9595d514a470'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:51.163253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284f5f828cbd4884b36f9595d514a470'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:51.163910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284f5f828cbd4884b36f9595d514a470'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:51.235661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284f5f828cbd4884b36f9595d514a470'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:51.256010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 930/1866] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:50 2022] DELETE /identity/v3/users/284f5f828cbd4884b36f9595d514a470 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:51.260552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8779d-269c-4714-a519-4e65f6fd5e34 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:51.260842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8779d-269c-4714-a519-4e65f6fd5e34 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:51.261117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8779d-269c-4714-a519-4e65f6fd5e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:51.281120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-9cd8779d-269c-4714-a519-4e65f6fd5e34 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:51.282336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9cd8779d-269c-4714-a519-4e65f6fd5e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:51.353637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 937/1867] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:51.363401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:51.364798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:51.425016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1151142-b4f9-45c6-be36-58668680011d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:51.429354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873a2e7-f420-46cb-a2dd-13605912108e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:51.430021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a2e7-f420-46cb-a2dd-13605912108e service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:51.434766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a2e7-f420-46cb-a2dd-13605912108e service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:51.434766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a2e7-f420-46cb-a2dd-13605912108e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:51.440841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873a2e7-f420-46cb-a2dd-13605912108e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:51.442317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873a2e7-f420-46cb-a2dd-13605912108e service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:51.453757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 931/1868] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:51.877225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8779d-269c-4714-a519-4e65f6fd5e34 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:51.879468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9cd8779d-269c-4714-a519-4e65f6fd5e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:51.898249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8779d-269c-4714-a519-4e65f6fd5e34 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:51.903832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:51.904368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:51.904649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:51.904920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] PATH_INFO: `/v3/projects/344787111544467a890e2c9bf4c2185f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:51.906075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=344787111544467a890e2c9bf4c2185f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:51.908075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:52.111253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344787111544467a890e2c9bf4c2185f', 'request_id': 'req-c803e17e-aaad-4691-a086-02730e44b4c7'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:52.116728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344787111544467a890e2c9bf4c2185f', 'request_id': 'req-c803e17e-aaad-4691-a086-02730e44b4c7'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:52.191840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344787111544467a890e2c9bf4c2185f', 'request_id': 'req-c803e17e-aaad-4691-a086-02730e44b4c7'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:52.204567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] The token cache is being invalidate because project 344787111544467a890e2c9bf4c2185f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:52.204851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:52.205481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c803e17e-aaad-4691-a086-02730e44b4c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:52.207083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 938/1869] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:51 2022] DELETE /identity/v3/projects/344787111544467a890e2c9bf4c2185f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:52.211971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78441742-3564-444c-b803-399bf010a61e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.212443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78441742-3564-444c-b803-399bf010a61e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.212531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78441742-3564-444c-b803-399bf010a61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:52.232026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-78441742-3564-444c-b803-399bf010a61e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:52.233225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-78441742-3564-444c-b803-399bf010a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:52.251033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:52.293246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 932/1870] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:52.302528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e638-2366-49a8-a491-7db8c59938f4 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.302801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e638-2366-49a8-a491-7db8c59938f4 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.303300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e638-2366-49a8-a491-7db8c59938f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:52.316219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:52.329112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8969e638-2366-49a8-a491-7db8c59938f4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:52.330314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8969e638-2366-49a8-a491-7db8c59938f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:52.338506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 939/1871] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:52.343007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78441742-3564-444c-b803-399bf010a61e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:52.355827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78441742-3564-444c-b803-399bf010a61e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:52.361521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:52.362259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.362387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.362641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:52.363354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:52.363651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:52.365359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:52.378503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 933/1872] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:52.382150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969e638-2366-49a8-a491-7db8c59938f4 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:52.393529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969e638-2366-49a8-a491-7db8c59938f4 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:52.398879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acf8c7-02ce-47bb-91cc-8b16a96e60ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:52.400540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acf8c7-02ce-47bb-91cc-8b16a96e60ca admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.400704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acf8c7-02ce-47bb-91cc-8b16a96e60ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.400977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acf8c7-02ce-47bb-91cc-8b16a96e60ca admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:52.402677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acf8c7-02ce-47bb-91cc-8b16a96e60ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:52.403614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acf8c7-02ce-47bb-91cc-8b16a96e60ca admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:52.521270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 940/1873] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:52 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:52.525356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:52.536139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ee82e-fd78-44ac-b74a-26a91dd1a21b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:52.604434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022985a-530e-461d-88bc-15c10b638dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:52.605057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022985a-530e-461d-88bc-15c10b638dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.605413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022985a-530e-461d-88bc-15c10b638dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.605698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022985a-530e-461d-88bc-15c10b638dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:52.606508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022985a-530e-461d-88bc-15c10b638dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:52.608986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022985a-530e-461d-88bc-15c10b638dc3 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:52.723901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 934/1874] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:52 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:52.727840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acf8c7-02ce-47bb-91cc-8b16a96e60ca admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:52.738200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acf8c7-02ce-47bb-91cc-8b16a96e60ca admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:52.743037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921ad1-fe29-4318-ad3d-45679fb834a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:52.743639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921ad1-fe29-4318-ad3d-45679fb834a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.743929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921ad1-fe29-4318-ad3d-45679fb834a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.744195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921ad1-fe29-4318-ad3d-45679fb834a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:52.744834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d921ad1-fe29-4318-ad3d-45679fb834a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:52.746433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d921ad1-fe29-4318-ad3d-45679fb834a2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:52.757264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 941/1875] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:52.761294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022985a-530e-461d-88bc-15c10b638dc3 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:52.772047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022985a-530e-461d-88bc-15c10b638dc3 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:52.776683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363fe0b-3e01-40d1-9d49-4c42e62a4ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:52.777271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363fe0b-3e01-40d1-9d49-4c42e62a4ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.777542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363fe0b-3e01-40d1-9d49-4c42e62a4ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.777809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363fe0b-3e01-40d1-9d49-4c42e62a4ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:52.778461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363fe0b-3e01-40d1-9d49-4c42e62a4ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:52.780065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363fe0b-3e01-40d1-9d49-4c42e62a4ce2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:52.790625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 935/1876] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:52.794752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921ad1-fe29-4318-ad3d-45679fb834a2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:52.804859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921ad1-fe29-4318-ad3d-45679fb834a2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:52.810579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:52.812838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.812838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.812838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] PATH_INFO: `/v3/projects/328b14f81aaf431784779b195c53e162/users/3c352c58b24248c2860f349fe7733ab8/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:52.828407 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=328b14f81aaf431784779b195c53e162, user_id=3c352c58b24248c2860f349fe7733ab8, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:52.831167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:52.987304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 942/1877] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:52 2022] PUT /identity/v3/projects/328b14f81aaf431784779b195c53e162/users/3c352c58b24248c2860f349fe7733ab8/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:52.998892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e77a3-5788-4385-9f00-1cdf979983b8 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:52.999639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e77a3-5788-4385-9f00-1cdf979983b8 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:52.999639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e77a3-5788-4385-9f00-1cdf979983b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:53.032100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-800e77a3-5788-4385-9f00-1cdf979983b8 None None] MFA Rules not processed for user `3c352c58b24248c2860f349fe7733ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:53.033471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-800e77a3-5788-4385-9f00-1cdf979983b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:53.112356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 936/1878] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:53.126285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:53.128313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:53.204112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4e2f-6c94-4bde-ad23-536ddd7fd2e1 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:53.208340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce466cd-5670-4193-88c6-b992e6cb6900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:53.209343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce466cd-5670-4193-88c6-b992e6cb6900 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:53.209870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce466cd-5670-4193-88c6-b992e6cb6900 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:53.210195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce466cd-5670-4193-88c6-b992e6cb6900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:53.220295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce466cd-5670-4193-88c6-b992e6cb6900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:53.221903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce466cd-5670-4193-88c6-b992e6cb6900 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:53.236076 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 943/1879] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:53.993161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e77a3-5788-4385-9f00-1cdf979983b8 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:53.994615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-800e77a3-5788-4385-9f00-1cdf979983b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:54.016034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:54.070574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e77a3-5788-4385-9f00-1cdf979983b8 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:54.076052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:54.076701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:54.077544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:54.077544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] PATH_INFO: `/v3/users/8654773473824a828f01c3ac22563ee3` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:54.078631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8654773473824a828f01c3ac22563ee3)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:54.080601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:54.296235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8654773473824a828f01c3ac22563ee3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:54.296941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8654773473824a828f01c3ac22563ee3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:54.314774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8654773473824a828f01c3ac22563ee3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:54.315619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8654773473824a828f01c3ac22563ee3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:54.426231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8654773473824a828f01c3ac22563ee3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:54.440911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 937/1880] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:53 2022] DELETE /identity/v3/users/8654773473824a828f01c3ac22563ee3 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:54.738365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce466cd-5670-4193-88c6-b992e6cb6900 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:54.750826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce466cd-5670-4193-88c6-b992e6cb6900 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:54.813421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:54.814085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:54.815143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:54.815425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] PATH_INFO: `/v3/projects/a04a23a1c79144a7a9690ec900cf9de2` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:54.817071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04a23a1c79144a7a9690ec900cf9de2)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:54.819349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:54.945728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:54.949230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04a23a1c79144a7a9690ec900cf9de2', 'request_id': 'req-06b1a951-b1a7-493f-88a7-05c58bcd9960'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:54.956878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04a23a1c79144a7a9690ec900cf9de2', 'request_id': 'req-06b1a951-b1a7-493f-88a7-05c58bcd9960'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:54.994190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:54.995962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:55.009613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2ee7e-06ac-496c-b981-0e4e55bc64a8 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:55.056715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04a23a1c79144a7a9690ec900cf9de2', 'request_id': 'req-06b1a951-b1a7-493f-88a7-05c58bcd9960'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.071897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] The token cache is being invalidate because project a04a23a1c79144a7a9690ec900cf9de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:55.072192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.073060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.075665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 944/1881] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:54 2022] DELETE /identity/v3/projects/a04a23a1c79144a7a9690ec900cf9de2 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:55.079796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:55.081324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:55.093596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:55.094421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:55.094857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:55.099326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] PATH_INFO: `/v3/users/32fcbaefaa6542639d840c982d7366b0` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:55.100789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fcbaefaa6542639d840c982d7366b0)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:55.103174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:55.156647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1a951-b1a7-493f-88a7-05c58bcd9960 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:55.161520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:55.162183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:55.162464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:55.162734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] PATH_INFO: `/v3/users/53e578d942d94258adebaa5b4b29d8f8` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:55.163885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e578d942d94258adebaa5b4b29d8f8)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:55.165868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:55.344229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:55.347942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e578d942d94258adebaa5b4b29d8f8'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.348965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e578d942d94258adebaa5b4b29d8f8'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.352788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fcbaefaa6542639d840c982d7366b0'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.353952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fcbaefaa6542639d840c982d7366b0'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.365732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e578d942d94258adebaa5b4b29d8f8'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.380482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fcbaefaa6542639d840c982d7366b0'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.381243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fcbaefaa6542639d840c982d7366b0'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.477669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e578d942d94258adebaa5b4b29d8f8'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.485159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b5c5e4de-6c48-4ef3-8355-d580f81b9638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e578d942d94258adebaa5b4b29d8f8'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.491336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 945/1882] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:55 2022] DELETE /identity/v3/users/53e578d942d94258adebaa5b4b29d8f8 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:55.568606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fcbaefaa6542639d840c982d7366b0'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.583257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 938/1883] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:54 2022] DELETE /identity/v3/users/32fcbaefaa6542639d840c982d7366b0 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:55.587718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222040-3367-4770-8eee-b616e320f22b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:55.588000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222040-3367-4770-8eee-b616e320f22b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:55.590079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222040-3367-4770-8eee-b616e320f22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:55.608749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8f222040-3367-4770-8eee-b616e320f22b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:55.610018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8f222040-3367-4770-8eee-b616e320f22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:55.666391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:55.678212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 946/1884] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:55.688716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:55.690112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:55.747855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6922b2-ec44-48f8-9551-362d1f7a73d7 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:55.751748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b5e49-d62c-4dc3-a259-8dc0e236c09a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:55.752350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b5e49-d62c-4dc3-a259-8dc0e236c09a service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:55.752623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b5e49-d62c-4dc3-a259-8dc0e236c09a service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:55.752891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b5e49-d62c-4dc3-a259-8dc0e236c09a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:55.762973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b5e49-d62c-4dc3-a259-8dc0e236c09a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:55.764508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b5e49-d62c-4dc3-a259-8dc0e236c09a service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:55.774381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 939/1885] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:55.804660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222040-3367-4770-8eee-b616e320f22b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:55.806137 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8f222040-3367-4770-8eee-b616e320f22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:55.819778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222040-3367-4770-8eee-b616e320f22b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:55.827753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:55.831083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:55.831083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:55.831083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] PATH_INFO: `/v3/projects/8a93edea26904dda911fc965fdbd41b0` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:55.832859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a93edea26904dda911fc965fdbd41b0)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:55.835665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:55.953233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a93edea26904dda911fc965fdbd41b0', 'request_id': 'req-3d401a7f-9eb2-46cb-816f-d98436a71a49'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:55.959020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a93edea26904dda911fc965fdbd41b0', 'request_id': 'req-3d401a7f-9eb2-46cb-816f-d98436a71a49'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.072343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a93edea26904dda911fc965fdbd41b0', 'request_id': 'req-3d401a7f-9eb2-46cb-816f-d98436a71a49'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.086738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] The token cache is being invalidate because project 8a93edea26904dda911fc965fdbd41b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:56.087055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.087730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3d401a7f-9eb2-46cb-816f-d98436a71a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.089258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 947/1886] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:55 2022] DELETE /identity/v3/projects/8a93edea26904dda911fc965fdbd41b0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:56.094843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447f835-9583-4c5d-b105-e7761d3c22e5 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.095325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447f835-9583-4c5d-b105-e7761d3c22e5 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.095559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447f835-9583-4c5d-b105-e7761d3c22e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.116742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0447f835-9583-4c5d-b105-e7761d3c22e5 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:56.118060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0447f835-9583-4c5d-b105-e7761d3c22e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.182546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 940/1887] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:56.190760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c35586-3752-4a78-afa3-b1a43727c74b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.191072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c35586-3752-4a78-afa3-b1a43727c74b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.191344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c35586-3752-4a78-afa3-b1a43727c74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.212100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-43c35586-3752-4a78-afa3-b1a43727c74b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:56.213287 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-43c35586-3752-4a78-afa3-b1a43727c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.221858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 948/1888] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:56.226353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447f835-9583-4c5d-b105-e7761d3c22e5 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.227695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c35586-3752-4a78-afa3-b1a43727c74b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.229035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-43c35586-3752-4a78-afa3-b1a43727c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.242849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447f835-9583-4c5d-b105-e7761d3c22e5 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.243346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c35586-3752-4a78-afa3-b1a43727c74b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.248419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.248580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.249145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.249309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.249404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.249661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] PATH_INFO: `/v3/projects/1841f10ca42740c3a3b69fed3e15ec51` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.249826 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.250299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.250777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1841f10ca42740c3a3b69fed3e15ec51)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.251163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:56.251561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.253057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.253607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.268385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 941/1889] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:56.273074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.286103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a628112-c0e0-466f-90b6-31e8f4b3dcb8 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.290896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.291587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.291813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.292083 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.292703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.294994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.369889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1841f10ca42740c3a3b69fed3e15ec51', 'request_id': 'req-67920ee1-ca8a-483d-96a9-9853278134fa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.379132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1841f10ca42740c3a3b69fed3e15ec51', 'request_id': 'req-67920ee1-ca8a-483d-96a9-9853278134fa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.400657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 942/1890] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:56 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:56.405466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.407267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.435431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:56.472720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0b8ff-a3a9-470e-b63d-cb49195d3c52 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.477401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.478027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.478302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.478574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] PATH_INFO: `/v3/users/3c352c58b24248c2860f349fe7733ab8` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.479823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c352c58b24248c2860f349fe7733ab8)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.482585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.509743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1841f10ca42740c3a3b69fed3e15ec51', 'request_id': 'req-67920ee1-ca8a-483d-96a9-9853278134fa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.522445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] The token cache is being invalidate because project 1841f10ca42740c3a3b69fed3e15ec51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:56.522784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.523541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.525037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 949/1891] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:56 2022] DELETE /identity/v3/projects/1841f10ca42740c3a3b69fed3e15ec51 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:56.526412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.527852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.583060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:56.595277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67920ee1-ca8a-483d-96a9-9853278134fa admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.600186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fada1-dfd3-4ff1-adef-93ee117dc349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.600888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fada1-dfd3-4ff1-adef-93ee117dc349 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.601170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fada1-dfd3-4ff1-adef-93ee117dc349 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.601437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fada1-dfd3-4ff1-adef-93ee117dc349 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.602090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fada1-dfd3-4ff1-adef-93ee117dc349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.604069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fada1-dfd3-4ff1-adef-93ee117dc349 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.685560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c352c58b24248c2860f349fe7733ab8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.686261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c352c58b24248c2860f349fe7733ab8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.698338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c352c58b24248c2860f349fe7733ab8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.699117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c352c58b24248c2860f349fe7733ab8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.750548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 950/1892] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:56 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:56.752181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e256-fd06-44e3-89e7-200715c323b2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.752463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e256-fd06-44e3-89e7-200715c323b2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.752857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e256-fd06-44e3-89e7-200715c323b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.772188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6758e256-fd06-44e3-89e7-200715c323b2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:56.773417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6758e256-fd06-44e3-89e7-200715c323b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.782452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 951/1893] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:56.783333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e256-fd06-44e3-89e7-200715c323b2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.784531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6758e256-fd06-44e3-89e7-200715c323b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.796699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e256-fd06-44e3-89e7-200715c323b2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.801361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542b456-4d1c-4ca8-934a-55d396f5df2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.801952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542b456-4d1c-4ca8-934a-55d396f5df2d admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.802231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542b456-4d1c-4ca8-934a-55d396f5df2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.802504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542b456-4d1c-4ca8-934a-55d396f5df2d admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.803167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542b456-4d1c-4ca8-934a-55d396f5df2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.804751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542b456-4d1c-4ca8-934a-55d396f5df2d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.809307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c352c58b24248c2860f349fe7733ab8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:56.816459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 952/1894] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:56.816459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab832-2bef-486a-9247-0040c65c2792 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.816749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab832-2bef-486a-9247-0040c65c2792 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.823210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab832-2bef-486a-9247-0040c65c2792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.825596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 943/1895] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:56 2022] DELETE /identity/v3/users/3c352c58b24248c2860f349fe7733ab8 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:56.826775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.837267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-984ab832-2bef-486a-9247-0040c65c2792 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:56.838410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-984ab832-2bef-486a-9247-0040c65c2792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.895231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c96186-c350-453d-9d93-b409e7c93fd1 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.899858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee962-2aad-49a7-a778-9ed83e05b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.900458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee962-2aad-49a7-a778-9ed83e05b6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.900727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee962-2aad-49a7-a778-9ed83e05b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.901004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee962-2aad-49a7-a778-9ed83e05b6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.901634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ee962-2aad-49a7-a778-9ed83e05b6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.904042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ee962-2aad-49a7-a778-9ed83e05b6d5 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.905544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 953/1896] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:56.906498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f0b24-90d2-4395-85ab-bba56a584104 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.906786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f0b24-90d2-4395-85ab-bba56a584104 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.907090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f0b24-90d2-4395-85ab-bba56a584104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.914721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 944/1897] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:56.915607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee962-2aad-49a7-a778-9ed83e05b6d5 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.925452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee962-2aad-49a7-a778-9ed83e05b6d5 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.930206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.930677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-f34f0b24-90d2-4395-85ab-bba56a584104 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:56.930801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.931249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.931526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.931928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f34f0b24-90d2-4395-85ab-bba56a584104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.932169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:56.932471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.934143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.940229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 954/1898] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:56.941006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f0b24-90d2-4395-85ab-bba56a584104 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.949043 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 945/1899] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:56.952076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:56.953936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:56.955956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f0b24-90d2-4395-85ab-bba56a584104 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:56.960820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4670d1-d9a7-4968-b232-480f6288ed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:56.961831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4670d1-d9a7-4968-b232-480f6288ed78 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:56.962808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4670d1-d9a7-4968-b232-480f6288ed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:56.965586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4670d1-d9a7-4968-b232-480f6288ed78 admin admin] PATH_INFO: `/v3/projects/5cd4fc25c3794f3cb789d8a00619a7f4/users/f8cb9ce02aa04cdab07f2c429ddd450a/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:56.979435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4670d1-d9a7-4968-b232-480f6288ed78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd4fc25c3794f3cb789d8a00619a7f4, user_id=f8cb9ce02aa04cdab07f2c429ddd450a, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:56.982236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4670d1-d9a7-4968-b232-480f6288ed78 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:56.984971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:57.010410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02541f-9fe5-4e51-b4c6-5b814daf456a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.014769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a3e22-35a5-4cca-a9f4-5ae0d538db10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.015398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a3e22-35a5-4cca-a9f4-5ae0d538db10 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.015679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a3e22-35a5-4cca-a9f4-5ae0d538db10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.015939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a3e22-35a5-4cca-a9f4-5ae0d538db10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.024866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a3e22-35a5-4cca-a9f4-5ae0d538db10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.026296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a3e22-35a5-4cca-a9f4-5ae0d538db10 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.037047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 946/1900] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:57.037838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a3e22-35a5-4cca-a9f4-5ae0d538db10 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.047533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a3e22-35a5-4cca-a9f4-5ae0d538db10 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.052275 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70fa9d-f279-4b6f-a21c-6c4b7cd22e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.052944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70fa9d-f279-4b6f-a21c-6c4b7cd22e39 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.053226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70fa9d-f279-4b6f-a21c-6c4b7cd22e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.053498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70fa9d-f279-4b6f-a21c-6c4b7cd22e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.054116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70fa9d-f279-4b6f-a21c-6c4b7cd22e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.056086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70fa9d-f279-4b6f-a21c-6c4b7cd22e39 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.110560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 955/1901] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:56 2022] PUT /identity/v3/projects/5cd4fc25c3794f3cb789d8a00619a7f4/users/f8cb9ce02aa04cdab07f2c429ddd450a/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:57.122508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466460e-3852-45a5-840c-be600b45c673 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.122881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466460e-3852-45a5-840c-be600b45c673 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.123243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466460e-3852-45a5-840c-be600b45c673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.151481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e466460e-3852-45a5-840c-be600b45c673 None None] MFA Rules not processed for user `f8cb9ce02aa04cdab07f2c429ddd450a`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:57.152784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e466460e-3852-45a5-840c-be600b45c673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:57.220426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 956/1902] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:57.231840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466460e-3852-45a5-840c-be600b45c673 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.233216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e466460e-3852-45a5-840c-be600b45c673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:57.239493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 947/1903] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:57 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:57.248370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70fa9d-f279-4b6f-a21c-6c4b7cd22e39 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.263080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:57.299957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466460e-3852-45a5-840c-be600b45c673 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.316674 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70fa9d-f279-4b6f-a21c-6c4b7cd22e39 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.321597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79448ea-325d-44ba-ab83-f28066e8eb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.322202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79448ea-325d-44ba-ab83-f28066e8eb2f admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.322481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79448ea-325d-44ba-ab83-f28066e8eb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.322749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79448ea-325d-44ba-ab83-f28066e8eb2f admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.323403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79448ea-325d-44ba-ab83-f28066e8eb2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.326434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79448ea-325d-44ba-ab83-f28066e8eb2f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.357386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38083a02-568e-416c-b7ee-8401af9cd80d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.361917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38083a02-568e-416c-b7ee-8401af9cd80d service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.361917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38083a02-568e-416c-b7ee-8401af9cd80d service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.361917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38083a02-568e-416c-b7ee-8401af9cd80d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.420994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38083a02-568e-416c-b7ee-8401af9cd80d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.421794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38083a02-568e-416c-b7ee-8401af9cd80d service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.431560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 957/1904] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:57:57.445458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 948/1905] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:57 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:57:57.460019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38083a02-568e-416c-b7ee-8401af9cd80d service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.460163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79448ea-325d-44ba-ab83-f28066e8eb2f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.471686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38083a02-568e-416c-b7ee-8401af9cd80d service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.473295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79448ea-325d-44ba-ab83-f28066e8eb2f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.478236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568fdd76-2be7-4fa9-abb0-c39dd2341620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.478850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fdd76-2be7-4fa9-abb0-c39dd2341620 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.479160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fdd76-2be7-4fa9-abb0-c39dd2341620 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.479261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.479428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fdd76-2be7-4fa9-abb0-c39dd2341620 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.479913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.480101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568fdd76-2be7-4fa9-abb0-c39dd2341620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.480209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.480481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] PATH_INFO: `/v3/users/f8cb9ce02aa04cdab07f2c429ddd450a` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.481780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8cb9ce02aa04cdab07f2c429ddd450a)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.481945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568fdd76-2be7-4fa9-abb0-c39dd2341620 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.483862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.495483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 949/1906] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:57.497072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568fdd76-2be7-4fa9-abb0-c39dd2341620 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.509277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568fdd76-2be7-4fa9-abb0-c39dd2341620 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.514864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e30bcc-a3c3-4571-adbe-0ec704b0ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.515504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e30bcc-a3c3-4571-adbe-0ec704b0ca75 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.515777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e30bcc-a3c3-4571-adbe-0ec704b0ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.518589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e30bcc-a3c3-4571-adbe-0ec704b0ca75 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.518589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e30bcc-a3c3-4571-adbe-0ec704b0ca75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.518589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e30bcc-a3c3-4571-adbe-0ec704b0ca75 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.529467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 950/1907] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:57:57.533495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e30bcc-a3c3-4571-adbe-0ec704b0ca75 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.547407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e30bcc-a3c3-4571-adbe-0ec704b0ca75 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.552597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.553712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.553951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.554223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] PATH_INFO: `/v3/projects/2ff1702cbe6d41b196df47d78f148368/users/abe845d55b4b4ac898556dbdc264c086/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.571534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff1702cbe6d41b196df47d78f148368, user_id=abe845d55b4b4ac898556dbdc264c086, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.576095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.659101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 951/1908] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:57 2022] PUT /identity/v3/projects/2ff1702cbe6d41b196df47d78f148368/users/abe845d55b4b4ac898556dbdc264c086/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:57.662702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.662702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:57.716238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:57.719437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8cb9ce02aa04cdab07f2c429ddd450a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.720108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8cb9ce02aa04cdab07f2c429ddd450a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.728417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f669c90-c8b2-41f6-b5e8-53f13b23afa3 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:57.732964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8cb9ce02aa04cdab07f2c429ddd450a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.733610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:57.734259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.734530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.734796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] PATH_INFO: `/v3/projects/328b14f81aaf431784779b195c53e162` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.735963 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=328b14f81aaf431784779b195c53e162)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:57.737840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:57.797843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8cb9ce02aa04cdab07f2c429ddd450a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.805849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7a4a83b4-bcb7-4597-84a0-958b629b2a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8cb9ce02aa04cdab07f2c429ddd450a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.813424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 958/1909] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:57:57 2022] DELETE /identity/v3/users/f8cb9ce02aa04cdab07f2c429ddd450a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:57.814564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e996f-2478-4acd-8207-b889d6330cb5 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.814907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e996f-2478-4acd-8207-b889d6330cb5 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.815270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e996f-2478-4acd-8207-b889d6330cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.829292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:57.836894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-328e996f-2478-4acd-8207-b889d6330cb5 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:57.838299 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-328e996f-2478-4acd-8207-b889d6330cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:57.848108 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328b14f81aaf431784779b195c53e162', 'request_id': 'req-0099181e-76ba-47ef-b393-ee01cebb5c31'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.854049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328b14f81aaf431784779b195c53e162', 'request_id': 'req-0099181e-76ba-47ef-b393-ee01cebb5c31'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.901174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 959/1910] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:57.902575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2075-3b1c-4096-8c64-f32a5acf123a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:57.903336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2075-3b1c-4096-8c64-f32a5acf123a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:57.903621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2075-3b1c-4096-8c64-f32a5acf123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:57.923409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-542d2075-3b1c-4096-8c64-f32a5acf123a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:57.924631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-542d2075-3b1c-4096-8c64-f32a5acf123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:57.932333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 960/1911] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:57.939109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2075-3b1c-4096-8c64-f32a5acf123a None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.939109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-542d2075-3b1c-4096-8c64-f32a5acf123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:57.957799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328b14f81aaf431784779b195c53e162', 'request_id': 'req-0099181e-76ba-47ef-b393-ee01cebb5c31'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.971866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] The token cache is being invalidate because project 328b14f81aaf431784779b195c53e162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:57.973614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.973614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:57.974419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 952/1912] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:57 2022] DELETE /identity/v3/projects/328b14f81aaf431784779b195c53e162 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:57.975212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:57.976485 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:58.001787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2075-3b1c-4096-8c64-f32a5acf123a None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.005390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb2883-5fe8-463c-a7dc-35f330640ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.005996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb2883-5fe8-463c-a7dc-35f330640ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.006270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb2883-5fe8-463c-a7dc-35f330640ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.006532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb2883-5fe8-463c-a7dc-35f330640ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.008322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-bceb2883-5fe8-463c-a7dc-35f330640ed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:58.020091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb2883-5fe8-463c-a7dc-35f330640ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.021508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb2883-5fe8-463c-a7dc-35f330640ed7 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.031768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 961/1913] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:58.033153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e4e63-7a5f-4330-b011-3bd60c2844af None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.033439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e4e63-7a5f-4330-b011-3bd60c2844af None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.033708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e4e63-7a5f-4330-b011-3bd60c2844af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.035235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099181e-76ba-47ef-b393-ee01cebb5c31 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.039773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.040394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.040811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.041089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.044789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:58.054888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-129e4e63-7a5f-4330-b011-3bd60c2844af None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:58.056042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-129e4e63-7a5f-4330-b011-3bd60c2844af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:58.059914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.061410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.064169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 962/1914] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:58.071853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28132d91-786f-4b5c-afbd-9ff48570f77a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.072130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28132d91-786f-4b5c-afbd-9ff48570f77a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.072398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28132d91-786f-4b5c-afbd-9ff48570f77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.072565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 953/1915] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:58.092875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-28132d91-786f-4b5c-afbd-9ff48570f77a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:58.096018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-28132d91-786f-4b5c-afbd-9ff48570f77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:58.103883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 963/1916] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:58.108942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:58.120574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f724de8-be87-41a8-963a-9fa9113458e0 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.127430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.128224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.128538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.128815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.129587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:58.129782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.131395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.144519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 954/1917] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:58.148821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28132d91-786f-4b5c-afbd-9ff48570f77a None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:58.159152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28132d91-786f-4b5c-afbd-9ff48570f77a None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.164366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487732c-707e-4800-8a01-496c744bc4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.164878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487732c-707e-4800-8a01-496c744bc4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.165404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487732c-707e-4800-8a01-496c744bc4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.165404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487732c-707e-4800-8a01-496c744bc4c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.166108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487732c-707e-4800-8a01-496c744bc4c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.168445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487732c-707e-4800-8a01-496c744bc4c9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.270131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 964/1918] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:58 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:58.274126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:58.285119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0749e1-70e3-4837-ac12-28d83a07a83a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.361578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f9580-cdbf-4195-b4c2-0b05e6559063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.362267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f9580-cdbf-4195-b4c2-0b05e6559063 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.362772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f9580-cdbf-4195-b4c2-0b05e6559063 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.363405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f9580-cdbf-4195-b4c2-0b05e6559063 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.363912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f9580-cdbf-4195-b4c2-0b05e6559063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.368138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f9580-cdbf-4195-b4c2-0b05e6559063 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.463113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:58.475302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 955/1919] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:58 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:58.477225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487732c-707e-4800-8a01-496c744bc4c9 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:58.491103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487732c-707e-4800-8a01-496c744bc4c9 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.499062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89333dc4-7d60-476e-8671-462e669ebef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.499900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89333dc4-7d60-476e-8671-462e669ebef2 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.500260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89333dc4-7d60-476e-8671-462e669ebef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.500868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89333dc4-7d60-476e-8671-462e669ebef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.501696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89333dc4-7d60-476e-8671-462e669ebef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.505078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89333dc4-7d60-476e-8671-462e669ebef2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.524435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 965/1920] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:58.526775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f9580-cdbf-4195-b4c2-0b05e6559063 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:58.540680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f9580-cdbf-4195-b4c2-0b05e6559063 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.546662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.547898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.548293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.548669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.549967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.552152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.576127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 956/1921] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:58.580816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89333dc4-7d60-476e-8671-462e669ebef2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:58.591755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89333dc4-7d60-476e-8671-462e669ebef2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.597489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e672dc-90da-4047-886b-ab61de9f16f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.598353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e672dc-90da-4047-886b-ab61de9f16f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.598353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e672dc-90da-4047-886b-ab61de9f16f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.598545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e672dc-90da-4047-886b-ab61de9f16f9 admin admin] PATH_INFO: `/v3/projects/535a526b62c547b5a05141ec01526a26/users/09ab9ac750494fc3b7930ea8c9e0e3c8/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.615145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e672dc-90da-4047-886b-ab61de9f16f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=535a526b62c547b5a05141ec01526a26, user_id=09ab9ac750494fc3b7930ea8c9e0e3c8, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.617948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e672dc-90da-4047-886b-ab61de9f16f9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.743451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 966/1922] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:58 2022] PUT /identity/v3/projects/535a526b62c547b5a05141ec01526a26/users/09ab9ac750494fc3b7930ea8c9e0e3c8/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:57:58.757161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:58.759166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:58.762623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f90714-831e-4f0a-a5fa-620bf937204d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.762901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f90714-831e-4f0a-a5fa-620bf937204d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.763202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f90714-831e-4f0a-a5fa-620bf937204d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.790163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fa889-896a-4265-a153-8f999f4d1c52 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:58.824668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d8f90714-831e-4f0a-a5fa-620bf937204d None None] MFA Rules not processed for user `09ab9ac750494fc3b7930ea8c9e0e3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:58.826422 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d8f90714-831e-4f0a-a5fa-620bf937204d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:58.855202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:58.878825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:58.879498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:58.879886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:58.880153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] PATH_INFO: `/v3/projects/5cd4fc25c3794f3cb789d8a00619a7f4` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:58.881283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd4fc25c3794f3cb789d8a00619a7f4)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:58.883210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:58.920091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 967/1923] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:58.932051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f90714-831e-4f0a-a5fa-620bf937204d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:58.933814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d8f90714-831e-4f0a-a5fa-620bf937204d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:59.009612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd4fc25c3794f3cb789d8a00619a7f4', 'request_id': 'req-9ed579df-03a9-4457-a11a-025d828a6a57'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:59.015136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd4fc25c3794f3cb789d8a00619a7f4', 'request_id': 'req-9ed579df-03a9-4457-a11a-025d828a6a57'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:59.025230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f90714-831e-4f0a-a5fa-620bf937204d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:59.081220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd4fc25c3794f3cb789d8a00619a7f4', 'request_id': 'req-9ed579df-03a9-4457-a11a-025d828a6a57'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:59.095213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f212cfd-9872-4717-b60e-da0b1d1b9de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:59.096034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212cfd-9872-4717-b60e-da0b1d1b9de6 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.096289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212cfd-9872-4717-b60e-da0b1d1b9de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.096692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212cfd-9872-4717-b60e-da0b1d1b9de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.099656 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] The token cache is being invalidate because project 5cd4fc25c3794f3cb789d8a00619a7f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:57:59.100022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:59.100709 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9ed579df-03a9-4457-a11a-025d828a6a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:57:59.107477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 957/1924] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:57:58 2022] DELETE /identity/v3/projects/5cd4fc25c3794f3cb789d8a00619a7f4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:59.110945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2172724-6279-4a1a-a87b-e6d644b81486 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.111302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2172724-6279-4a1a-a87b-e6d644b81486 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.111641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2172724-6279-4a1a-a87b-e6d644b81486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.164231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c2172724-6279-4a1a-a87b-e6d644b81486 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:59.167142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c2172724-6279-4a1a-a87b-e6d644b81486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:59.207839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f212cfd-9872-4717-b60e-da0b1d1b9de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:59.209401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f212cfd-9872-4717-b60e-da0b1d1b9de6 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:59.224064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 968/1925] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:57:59.275256 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 958/1926] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:57:59.283098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd1a0e-9858-40a7-a2b5-4b0fa5646ac8 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.283374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd1a0e-9858-40a7-a2b5-4b0fa5646ac8 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.283652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd1a0e-9858-40a7-a2b5-4b0fa5646ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.305872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-41dd1a0e-9858-40a7-a2b5-4b0fa5646ac8 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:59.307195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-41dd1a0e-9858-40a7-a2b5-4b0fa5646ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:59.316139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 969/1927] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:59.320710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2172724-6279-4a1a-a87b-e6d644b81486 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:59.332076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2172724-6279-4a1a-a87b-e6d644b81486 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:59.336789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:59.337395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.337673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.337947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.338643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:57:59.338971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:59.341297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:59.353357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 959/1928] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:57:59.358183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd1a0e-9858-40a7-a2b5-4b0fa5646ac8 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:59.367564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd1a0e-9858-40a7-a2b5-4b0fa5646ac8 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:59.372386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39104c13-30a0-43a1-9deb-9a6439d2a755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:59.373001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104c13-30a0-43a1-9deb-9a6439d2a755 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.373264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104c13-30a0-43a1-9deb-9a6439d2a755 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.373535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39104c13-30a0-43a1-9deb-9a6439d2a755 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.374197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39104c13-30a0-43a1-9deb-9a6439d2a755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:59.376168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39104c13-30a0-43a1-9deb-9a6439d2a755 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:59.485520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 970/1929] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:57:59 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:59.489724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:59.500889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc1647-1097-4dd6-8f64-2da94f80ebd1 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:59.504822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:59.557478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0ef14-ae2c-4887-a30d-4f670fc61ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:59.558074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0ef14-ae2c-4887-a30d-4f670fc61ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.558350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0ef14-ae2c-4887-a30d-4f670fc61ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.558622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0ef14-ae2c-4887-a30d-4f670fc61ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.559937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0ef14-ae2c-4887-a30d-4f670fc61ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:59.561895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0ef14-ae2c-4887-a30d-4f670fc61ad9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:59.679138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 960/1930] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:57:59 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:57:59.680377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39104c13-30a0-43a1-9deb-9a6439d2a755 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:59.691952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39104c13-30a0-43a1-9deb-9a6439d2a755 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:59.696614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94ea6-375d-4648-928a-fec6668fad10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:59.697226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94ea6-375d-4648-928a-fec6668fad10 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.697507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94ea6-375d-4648-928a-fec6668fad10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.698394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94ea6-375d-4648-928a-fec6668fad10 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.699097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94ea6-375d-4648-928a-fec6668fad10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:59.700768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94ea6-375d-4648-928a-fec6668fad10 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:59.714746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 971/1931] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:59.716680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0ef14-ae2c-4887-a30d-4f670fc61ad9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:59.729842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0ef14-ae2c-4887-a30d-4f670fc61ad9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:59.736942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bf6bf-43f4-45d1-8232-995a75bde032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:59.737598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bf6bf-43f4-45d1-8232-995a75bde032 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.745412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bf6bf-43f4-45d1-8232-995a75bde032 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.745412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bf6bf-43f4-45d1-8232-995a75bde032 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.745412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bf6bf-43f4-45d1-8232-995a75bde032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:59.745412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bf6bf-43f4-45d1-8232-995a75bde032 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:59.756291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 961/1932] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:57:59.757720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94ea6-375d-4648-928a-fec6668fad10 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:59.766248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:57:59.770383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94ea6-375d-4648-928a-fec6668fad10 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:57:59.775739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c75cbd-081f-4013-88f8-2a572231e549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:57:59.776810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c75cbd-081f-4013-88f8-2a572231e549 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.777096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c75cbd-081f-4013-88f8-2a572231e549 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.777373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c75cbd-081f-4013-88f8-2a572231e549 admin admin] PATH_INFO: `/v3/projects/377bc47cac8e4cd6821ecdb74755c741/users/88b66441b1434f94ba277ab46df963ac/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.795464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c75cbd-081f-4013-88f8-2a572231e549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=377bc47cac8e4cd6821ecdb74755c741, user_id=88b66441b1434f94ba277ab46df963ac, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:57:59.797947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c75cbd-081f-4013-88f8-2a572231e549 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:57:59.804827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f61a9-6512-4a68-9e54-161c43fc9202 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.805200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f61a9-6512-4a68-9e54-161c43fc9202 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.805544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f61a9-6512-4a68-9e54-161c43fc9202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.835630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f23f61a9-6512-4a68-9e54-161c43fc9202 None None] MFA Rules not processed for user `abe845d55b4b4ac898556dbdc264c086`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:59.836591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f23f61a9-6512-4a68-9e54-161c43fc9202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:59.890050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 972/1933] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:57:59 2022] PUT /identity/v3/projects/377bc47cac8e4cd6821ecdb74755c741/users/88b66441b1434f94ba277ab46df963ac/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:57:59.891803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad8f6-d9a8-4c6c-96bc-58307b7729ef None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:57:59.891914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad8f6-d9a8-4c6c-96bc-58307b7729ef None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:57:59.892189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad8f6-d9a8-4c6c-96bc-58307b7729ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:57:59.916921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 962/1934] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:57:59.940722 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-303ad8f6-d9a8-4c6c-96bc-58307b7729ef None None] MFA Rules not processed for user `88b66441b1434f94ba277ab46df963ac`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:57:59.942011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-303ad8f6-d9a8-4c6c-96bc-58307b7729ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:57:59.949429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f61a9-6512-4a68-9e54-161c43fc9202 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:57:59.951087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f23f61a9-6512-4a68-9e54-161c43fc9202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:00.046108 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f61a9-6512-4a68-9e54-161c43fc9202 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:00.052639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 973/1935] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:57:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:00.065043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad8f6-d9a8-4c6c-96bc-58307b7729ef None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:00.066560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-303ad8f6-d9a8-4c6c-96bc-58307b7729ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:00.117348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:00.117980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:00.118259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:00.118530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:00.128748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad8f6-d9a8-4c6c-96bc-58307b7729ef None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:00.132495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfbaf2-0941-4579-b6e9-46f9e53fbcf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:00.133134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfbaf2-0941-4579-b6e9-46f9e53fbcf1 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:00.133408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfbaf2-0941-4579-b6e9-46f9e53fbcf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:00.133670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfbaf2-0941-4579-b6e9-46f9e53fbcf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:00.142793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfbaf2-0941-4579-b6e9-46f9e53fbcf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:00.144325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfbaf2-0941-4579-b6e9-46f9e53fbcf1 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:00.154057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 974/1936] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:58:00.178895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:00.180725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:00.198490 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 963/1937] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:57:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:00.969322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:02.009580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6826d2-92a0-46a1-ae40-f33a18cc2932 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:02.010144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6826d2-92a0-46a1-ae40-f33a18cc2932 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:02.010144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6826d2-92a0-46a1-ae40-f33a18cc2932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:02.031280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7b6826d2-92a0-46a1-ae40-f33a18cc2932 None None] MFA Rules not processed for user `abe845d55b4b4ac898556dbdc264c086`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:02.032498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7b6826d2-92a0-46a1-ae40-f33a18cc2932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:02.039372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 975/1938] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:11.592247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=34 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:11.977905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:11.979829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:12.025697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:12.082243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e53c-d75d-46ab-bde3-fe6c0fe242c5 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:12.086902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:12.087589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:12.088296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:12.088575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] PATH_INFO: `/v3/users/f56fcb6b073d4f1d9dabab1e0bac6ec3` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:12.090082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56fcb6b073d4f1d9dabab1e0bac6ec3)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:12.092713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:12.380522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56fcb6b073d4f1d9dabab1e0bac6ec3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:12.381179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56fcb6b073d4f1d9dabab1e0bac6ec3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:12.393904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56fcb6b073d4f1d9dabab1e0bac6ec3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:12.395808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56fcb6b073d4f1d9dabab1e0bac6ec3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:12.475626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-155ca7fc-819b-4c1e-a539-b435284e7bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56fcb6b073d4f1d9dabab1e0bac6ec3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:12.492971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 964/1939] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:11 2022] DELETE /identity/v3/users/f56fcb6b073d4f1d9dabab1e0bac6ec3 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:12.829592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6826d2-92a0-46a1-ae40-f33a18cc2932 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:12.847308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6826d2-92a0-46a1-ae40-f33a18cc2932 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:12.914084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:12.914732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:12.915289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:12.916088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] PATH_INFO: `/v3/projects/e649b58115ab4bcbbf045a3c036a3049` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:12.917349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e649b58115ab4bcbbf045a3c036a3049)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:12.919449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:13.099853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e649b58115ab4bcbbf045a3c036a3049', 'request_id': 'req-f3029337-1a03-42da-b1c6-9650efd69a60'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:13.109303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e649b58115ab4bcbbf045a3c036a3049', 'request_id': 'req-f3029337-1a03-42da-b1c6-9650efd69a60'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:13.225362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e649b58115ab4bcbbf045a3c036a3049', 'request_id': 'req-f3029337-1a03-42da-b1c6-9650efd69a60'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:13.254610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] The token cache is being invalidate because project e649b58115ab4bcbbf045a3c036a3049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:13.254972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:13.255647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f3029337-1a03-42da-b1c6-9650efd69a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:13.257844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 976/1940] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:12 2022] DELETE /identity/v3/projects/e649b58115ab4bcbbf045a3c036a3049 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:13.269427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74da91-5e27-4d15-8d57-984e49089686 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:13.270036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74da91-5e27-4d15-8d57-984e49089686 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:13.270036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74da91-5e27-4d15-8d57-984e49089686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:13.291224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ac74da91-5e27-4d15-8d57-984e49089686 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:13.300888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ac74da91-5e27-4d15-8d57-984e49089686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:13.357884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 965/1941] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:13.376670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce2954-a2e3-4d50-814e-fa1412f80ed4 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:13.377018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce2954-a2e3-4d50-814e-fa1412f80ed4 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:13.377409 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce2954-a2e3-4d50-814e-fa1412f80ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:13.400781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-f1ce2954-a2e3-4d50-814e-fa1412f80ed4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:13.402361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f1ce2954-a2e3-4d50-814e-fa1412f80ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:13.412336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 977/1942] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:13.416613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74da91-5e27-4d15-8d57-984e49089686 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:13.427478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74da91-5e27-4d15-8d57-984e49089686 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:13.444036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:13.444790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:13.445173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:13.445445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:13.446099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:58:13.446400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:13.448086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:13.463846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 966/1943] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:58:13.468470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce2954-a2e3-4d50-814e-fa1412f80ed4 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:13.478487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce2954-a2e3-4d50-814e-fa1412f80ed4 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:13.484279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213dbab-db06-4482-885d-7d067853f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:13.484899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213dbab-db06-4482-885d-7d067853f671 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:13.485346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213dbab-db06-4482-885d-7d067853f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:13.485618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213dbab-db06-4482-885d-7d067853f671 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:13.486412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213dbab-db06-4482-885d-7d067853f671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:13.488512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213dbab-db06-4482-885d-7d067853f671 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:13.609059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 978/1944] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:13 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:13.610733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:13.622985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9b16b-1e58-4f44-9d6e-e907161a1a44 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:13.626692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:13.680214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fb28e-0a07-4f47-8578-8f9862486a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:13.680853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fb28e-0a07-4f47-8578-8f9862486a0b admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:13.681127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fb28e-0a07-4f47-8578-8f9862486a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:13.681393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fb28e-0a07-4f47-8578-8f9862486a0b admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:13.682008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fb28e-0a07-4f47-8578-8f9862486a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:13.684909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fb28e-0a07-4f47-8578-8f9862486a0b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:13.809358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 967/1945] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:13 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:13.811110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213dbab-db06-4482-885d-7d067853f671 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:13.821965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213dbab-db06-4482-885d-7d067853f671 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:13.827179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8528d7e-7ec1-44a3-aee4-1ba11563d560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:13.827801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8528d7e-7ec1-44a3-aee4-1ba11563d560 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:13.828075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8528d7e-7ec1-44a3-aee4-1ba11563d560 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:13.828346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8528d7e-7ec1-44a3-aee4-1ba11563d560 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:13.828990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8528d7e-7ec1-44a3-aee4-1ba11563d560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:13.830670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8528d7e-7ec1-44a3-aee4-1ba11563d560 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:13.842653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 979/1946] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:13.846494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fb28e-0a07-4f47-8578-8f9862486a0b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:13.857327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fb28e-0a07-4f47-8578-8f9862486a0b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:13.861938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c42082-c961-4f91-bc8e-38135e4e7231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:13.862564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42082-c961-4f91-bc8e-38135e4e7231 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:13.862839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42082-c961-4f91-bc8e-38135e4e7231 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:13.863208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42082-c961-4f91-bc8e-38135e4e7231 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:13.863844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c42082-c961-4f91-bc8e-38135e4e7231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:13.865416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c42082-c961-4f91-bc8e-38135e4e7231 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:13.877942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 968/1947] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:13.882249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8528d7e-7ec1-44a3-aee4-1ba11563d560 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:13.892747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8528d7e-7ec1-44a3-aee4-1ba11563d560 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:13.897764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:13.898598 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:13.898962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:13.899241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] PATH_INFO: `/v3/projects/ffe6501464cc4c45994be1b272106870/users/419e09ee94b14a639b0a0716e1decdc6/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:13.918617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe6501464cc4c45994be1b272106870, user_id=419e09ee94b14a639b0a0716e1decdc6, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:13.921747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:14.015235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 980/1948] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:13 2022] PUT /identity/v3/projects/ffe6501464cc4c45994be1b272106870/users/419e09ee94b14a639b0a0716e1decdc6/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:14.025566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3c0a8-9f05-4168-9ecf-2c5473291995 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:14.025847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3c0a8-9f05-4168-9ecf-2c5473291995 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:14.026113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3c0a8-9f05-4168-9ecf-2c5473291995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:14.054134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-b1f3c0a8-9f05-4168-9ecf-2c5473291995 None None] MFA Rules not processed for user `419e09ee94b14a639b0a0716e1decdc6`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:14.055368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b1f3c0a8-9f05-4168-9ecf-2c5473291995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:14.124187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 969/1949] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:14.134865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:14.137391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:14.171813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:14.215720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629bd88-c103-4f70-a6f8-3f8dc7397801 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:14.221209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ba1e2-390b-49b6-8918-b25dd1b690c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:14.221807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ba1e2-390b-49b6-8918-b25dd1b690c8 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:14.222080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ba1e2-390b-49b6-8918-b25dd1b690c8 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:14.222345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ba1e2-390b-49b6-8918-b25dd1b690c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:14.230158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ba1e2-390b-49b6-8918-b25dd1b690c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:14.231917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ba1e2-390b-49b6-8918-b25dd1b690c8 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:14.242270 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 981/1950] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:14.471917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3c0a8-9f05-4168-9ecf-2c5473291995 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:14.473842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b1f3c0a8-9f05-4168-9ecf-2c5473291995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:14.547518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3c0a8-9f05-4168-9ecf-2c5473291995 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:14.552426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:14.553068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:14.553342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:14.553610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] PATH_INFO: `/v3/users/09ab9ac750494fc3b7930ea8c9e0e3c8` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:14.554727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ab9ac750494fc3b7930ea8c9e0e3c8)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:14.556869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:14.797448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ab9ac750494fc3b7930ea8c9e0e3c8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:14.798118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ab9ac750494fc3b7930ea8c9e0e3c8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:14.810904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ab9ac750494fc3b7930ea8c9e0e3c8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:14.811625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ab9ac750494fc3b7930ea8c9e0e3c8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:14.911981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ab9ac750494fc3b7930ea8c9e0e3c8'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:14.927166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 970/1951] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:14 2022] DELETE /identity/v3/users/09ab9ac750494fc3b7930ea8c9e0e3c8 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:14.931675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dde97-6e45-4d9f-a9bb-976ded95c0b5 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:14.931962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dde97-6e45-4d9f-a9bb-976ded95c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:14.932234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dde97-6e45-4d9f-a9bb-976ded95c0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:14.953061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-457dde97-6e45-4d9f-a9bb-976ded95c0b5 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:14.954357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-457dde97-6e45-4d9f-a9bb-976ded95c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:15.022533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 982/1952] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:15.032738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:15.036158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:15.120452 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12cfcb-001c-4d36-8c9b-f4b6bf0fcd7d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:15.124517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb420c-ef2c-4a5b-abc8-665fadab9e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:15.125232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb420c-ef2c-4a5b-abc8-665fadab9e93 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:15.125510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb420c-ef2c-4a5b-abc8-665fadab9e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:15.125766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb420c-ef2c-4a5b-abc8-665fadab9e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:15.135998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb420c-ef2c-4a5b-abc8-665fadab9e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:15.138459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb420c-ef2c-4a5b-abc8-665fadab9e93 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:15.149975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 971/1953] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:15.760436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dde97-6e45-4d9f-a9bb-976ded95c0b5 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:15.763114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-457dde97-6e45-4d9f-a9bb-976ded95c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:15.776614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dde97-6e45-4d9f-a9bb-976ded95c0b5 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:15.787199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:15.788439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:15.788814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:15.789306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] PATH_INFO: `/v3/projects/535a526b62c547b5a05141ec01526a26` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:15.790569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=535a526b62c547b5a05141ec01526a26)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:15.792847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:15.937005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535a526b62c547b5a05141ec01526a26', 'request_id': 'req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:15.942897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535a526b62c547b5a05141ec01526a26', 'request_id': 'req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:15.946336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:16.077378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535a526b62c547b5a05141ec01526a26', 'request_id': 'req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:16.100614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] The token cache is being invalidate because project 535a526b62c547b5a05141ec01526a26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:16.100943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:16.101652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-30351842-7c0a-47a6-ae91-8fa0be8f3fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:16.103565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 983/1954] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:15 2022] DELETE /identity/v3/projects/535a526b62c547b5a05141ec01526a26 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:16.109230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796675b-1cbf-488c-b387-fc4da9609e18 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.109507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796675b-1cbf-488c-b387-fc4da9609e18 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.109774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796675b-1cbf-488c-b387-fc4da9609e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.129291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e796675b-1cbf-488c-b387-fc4da9609e18 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:16.130479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e796675b-1cbf-488c-b387-fc4da9609e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:16.189821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 972/1955] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:16.197640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3aee-df8b-47bd-9b39-e08b1311b6d3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.197915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3aee-df8b-47bd-9b39-e08b1311b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.198180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3aee-df8b-47bd-9b39-e08b1311b6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.219233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3c2e3aee-df8b-47bd-9b39-e08b1311b6d3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:16.220444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3c2e3aee-df8b-47bd-9b39-e08b1311b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:16.228182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 984/1956] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:16.232529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796675b-1cbf-488c-b387-fc4da9609e18 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:16.242641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796675b-1cbf-488c-b387-fc4da9609e18 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:16.247881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:16.248570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.248827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.249090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.249801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:58:16.250001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:16.251629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:16.271615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 973/1957] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:58:16.274372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e3aee-df8b-47bd-9b39-e08b1311b6d3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:16.284651 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e3aee-df8b-47bd-9b39-e08b1311b6d3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:16.289437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ff24-b510-43f9-a732-c8825c44b001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:16.290046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ff24-b510-43f9-a732-c8825c44b001 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.290334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ff24-b510-43f9-a732-c8825c44b001 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.290712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ff24-b510-43f9-a732-c8825c44b001 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.291367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13ff24-b510-43f9-a732-c8825c44b001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:16.293934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13ff24-b510-43f9-a732-c8825c44b001 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:16.442315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 985/1958] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:16 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:16.448491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:16.460654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960b960-a36b-47ef-906b-b9fc1b83ab48 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:16.526903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3aaf3-ec87-4256-a134-ba9ba3e42e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:16.527724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3aaf3-ec87-4256-a134-ba9ba3e42e59 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.528073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3aaf3-ec87-4256-a134-ba9ba3e42e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.528540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3aaf3-ec87-4256-a134-ba9ba3e42e59 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.529175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3aaf3-ec87-4256-a134-ba9ba3e42e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:16.531185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3aaf3-ec87-4256-a134-ba9ba3e42e59 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:16.654210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 974/1959] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:16 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:58:16.655620 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ff24-b510-43f9-a732-c8825c44b001 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:16.666947 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13ff24-b510-43f9-a732-c8825c44b001 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:16.672811 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a5d1d-2fb3-4b66-aa41-a963ce0f6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:16.673422 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a5d1d-2fb3-4b66-aa41-a963ce0f6ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.673771 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a5d1d-2fb3-4b66-aa41-a963ce0f6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.673945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a5d1d-2fb3-4b66-aa41-a963ce0f6ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.674591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a5d1d-2fb3-4b66-aa41-a963ce0f6ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:16.676248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a5d1d-2fb3-4b66-aa41-a963ce0f6ac9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:16.679030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:16.688606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 986/1960] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:58:16.692616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3aaf3-ec87-4256-a134-ba9ba3e42e59 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:16.703557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3aaf3-ec87-4256-a134-ba9ba3e42e59 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:16.708344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c4a89-bc03-4f36-b08c-409caa6a989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:16.708944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c4a89-bc03-4f36-b08c-409caa6a989c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.709217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c4a89-bc03-4f36-b08c-409caa6a989c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.709481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c4a89-bc03-4f36-b08c-409caa6a989c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.710114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c4a89-bc03-4f36-b08c-409caa6a989c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:16.711814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c4a89-bc03-4f36-b08c-409caa6a989c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:16.723730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 975/1961] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:16.728077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a5d1d-2fb3-4b66-aa41-a963ce0f6ac9 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:16.739203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a5d1d-2fb3-4b66-aa41-a963ce0f6ac9 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:16.744088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:16.744740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.745012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.745275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] PATH_INFO: `/v3/projects/f3a9b259e9d54603af734306adbfe16c/users/a0ecfd3d64ba4fef820a31b1522125ff/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.760371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a9b259e9d54603af734306adbfe16c, user_id=a0ecfd3d64ba4fef820a31b1522125ff, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:16.763569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:16.934969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 987/1962] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:16 2022] PUT /identity/v3/projects/f3a9b259e9d54603af734306adbfe16c/users/a0ecfd3d64ba4fef820a31b1522125ff/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:16.939155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0ff54-b52c-4868-a2d3-33b9101695c0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:16.939488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0ff54-b52c-4868-a2d3-33b9101695c0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:16.939762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0ff54-b52c-4868-a2d3-33b9101695c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:16.965198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-aff0ff54-b52c-4868-a2d3-33b9101695c0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:16.966903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-aff0ff54-b52c-4868-a2d3-33b9101695c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:17.033041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 976/1963] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:17.044428 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:17.046324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:17.107761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca0733-edf6-49d5-9b53-2bf2b42f89dc admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:17.111658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e146a7-5aa8-4324-a6a1-ce8e22ed02fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:17.112279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e146a7-5aa8-4324-a6a1-ce8e22ed02fe service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:17.112614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e146a7-5aa8-4324-a6a1-ce8e22ed02fe service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:17.112889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e146a7-5aa8-4324-a6a1-ce8e22ed02fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:17.124021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e146a7-5aa8-4324-a6a1-ce8e22ed02fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:17.125410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e146a7-5aa8-4324-a6a1-ce8e22ed02fe service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:17.135657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 988/1964] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:17.741442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0ff54-b52c-4868-a2d3-33b9101695c0 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:17.741442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-aff0ff54-b52c-4868-a2d3-33b9101695c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:17.754114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0ff54-b52c-4868-a2d3-33b9101695c0 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:17.758896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:17.759754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:17.760116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:17.760477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] PATH_INFO: `/v3/users/419e09ee94b14a639b0a0716e1decdc6` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:17.761879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] RBAC: Authorizing `identity:delete_user(user_id=419e09ee94b14a639b0a0716e1decdc6)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:17.764628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:18.200730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '419e09ee94b14a639b0a0716e1decdc6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:18.201645 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '419e09ee94b14a639b0a0716e1decdc6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:18.207594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:18.216734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '419e09ee94b14a639b0a0716e1decdc6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:18.217420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '419e09ee94b14a639b0a0716e1decdc6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:18.355110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '419e09ee94b14a639b0a0716e1decdc6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:18.369585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 977/1965] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:17 2022] DELETE /identity/v3/users/419e09ee94b14a639b0a0716e1decdc6 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:18.374167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7b283-06ac-4aef-8b5d-326f980fce6c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:18.374674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7b283-06ac-4aef-8b5d-326f980fce6c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:18.374766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7b283-06ac-4aef-8b5d-326f980fce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:18.396728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-77d7b283-06ac-4aef-8b5d-326f980fce6c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:18.397945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-77d7b283-06ac-4aef-8b5d-326f980fce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:18.463152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 989/1966] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:18.473906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:18.475376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:18.539455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e64f23-e194-476b-ae02-5cf2cee1641b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:18.543905 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851e6e9-0464-4b0b-9895-1d347d822d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:18.544570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e6e9-0464-4b0b-9895-1d347d822d6b service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:18.544848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e6e9-0464-4b0b-9895-1d347d822d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:18.545116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e6e9-0464-4b0b-9895-1d347d822d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:18.554234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851e6e9-0464-4b0b-9895-1d347d822d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:18.555729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851e6e9-0464-4b0b-9895-1d347d822d6b service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:18.566190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 978/1967] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:18.979263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7b283-06ac-4aef-8b5d-326f980fce6c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:18.984081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-77d7b283-06ac-4aef-8b5d-326f980fce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:18.994332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7b283-06ac-4aef-8b5d-326f980fce6c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:18.999666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:19.000396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.000743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.001091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] PATH_INFO: `/v3/projects/ffe6501464cc4c45994be1b272106870` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.002683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffe6501464cc4c45994be1b272106870)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:19.005120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:19.052602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71105a-e153-4135-9758-3b3bc1a88d8d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.052899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71105a-e153-4135-9758-3b3bc1a88d8d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.053173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71105a-e153-4135-9758-3b3bc1a88d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.093281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-9a71105a-e153-4135-9758-3b3bc1a88d8d None None] MFA Rules not processed for user `a0ecfd3d64ba4fef820a31b1522125ff`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:19.094636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9a71105a-e153-4135-9758-3b3bc1a88d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:19.147912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffe6501464cc4c45994be1b272106870', 'request_id': 'req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:19.163193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffe6501464cc4c45994be1b272106870', 'request_id': 'req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:19.185173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 979/1968] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:19.196010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71105a-e153-4135-9758-3b3bc1a88d8d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:19.197416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9a71105a-e153-4135-9758-3b3bc1a88d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:19.257284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71105a-e153-4135-9758-3b3bc1a88d8d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:19.260263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffe6501464cc4c45994be1b272106870', 'request_id': 'req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:19.268740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:19.273206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] The token cache is being invalidate because project ffe6501464cc4c45994be1b272106870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:19.273497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:19.274138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3f7c3c4e-f41f-40e9-8c36-1f10be7ff9fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:19.275973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 990/1969] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:18 2022] DELETE /identity/v3/projects/ffe6501464cc4c45994be1b272106870 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:19.280743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b252609-0fcb-44b0-99f4-d576e73afd8f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.281023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b252609-0fcb-44b0-99f4-d576e73afd8f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.281295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b252609-0fcb-44b0-99f4-d576e73afd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.302888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-0b252609-0fcb-44b0-99f4-d576e73afd8f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:19.304137 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0b252609-0fcb-44b0-99f4-d576e73afd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:19.309962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:19.310583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.310858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.311159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.312855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:19.364656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 991/1970] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:19.372990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019408f2-f1b1-4189-b3bc-54fbe9c9a555 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.373393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019408f2-f1b1-4189-b3bc-54fbe9c9a555 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.373675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019408f2-f1b1-4189-b3bc-54fbe9c9a555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.374369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:19.376076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:19.385497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 980/1971] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 15:58:19.394766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-019408f2-f1b1-4189-b3bc-54fbe9c9a555 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:19.395958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-019408f2-f1b1-4189-b3bc-54fbe9c9a555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:19.405554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 992/1972] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:19.415354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:19.428032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89571-7026-4805-8ee5-b461fd87662e service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:19.433053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:19.434005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.434351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.434697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.435491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:58:19.435791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:19.437428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:19.450236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 981/1973] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:58:19.466436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019408f2-f1b1-4189-b3bc-54fbe9c9a555 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:19.475538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019408f2-f1b1-4189-b3bc-54fbe9c9a555 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:19.480335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36e1f4-c6c0-4b7c-97ca-e45096b2e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:19.480938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36e1f4-c6c0-4b7c-97ca-e45096b2e50f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.481211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36e1f4-c6c0-4b7c-97ca-e45096b2e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.481560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36e1f4-c6c0-4b7c-97ca-e45096b2e50f admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.482190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36e1f4-c6c0-4b7c-97ca-e45096b2e50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:19.484113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36e1f4-c6c0-4b7c-97ca-e45096b2e50f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:19.526374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:19.582604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 993/1974] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:19 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:19.586849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:19.597223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bb2dc-bf0b-489d-ad08-1d761474ae7b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:19.656184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4a523-8865-400f-90e5-15610f79b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:19.656769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4a523-8865-400f-90e5-15610f79b0cf admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.657049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4a523-8865-400f-90e5-15610f79b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.657321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4a523-8865-400f-90e5-15610f79b0cf admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.657983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4a523-8865-400f-90e5-15610f79b0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:19.659926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4a523-8865-400f-90e5-15610f79b0cf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:19.801345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 982/1975] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:19 2022] POST /identity/v3/users => generated 279 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:19.805406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36e1f4-c6c0-4b7c-97ca-e45096b2e50f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:19.815696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36e1f4-c6c0-4b7c-97ca-e45096b2e50f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:19.820352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f6fb7-a5ac-4229-b4f1-fa1d926f894b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:19.820949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f6fb7-a5ac-4229-b4f1-fa1d926f894b admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.821232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f6fb7-a5ac-4229-b4f1-fa1d926f894b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.821501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f6fb7-a5ac-4229-b4f1-fa1d926f894b admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.822139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f6fb7-a5ac-4229-b4f1-fa1d926f894b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:19.823757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f6fb7-a5ac-4229-b4f1-fa1d926f894b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:19.835465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 994/1976] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:19.839374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4a523-8865-400f-90e5-15610f79b0cf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:19.850088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4a523-8865-400f-90e5-15610f79b0cf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:19.854889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e588d4e-dfb3-4c1d-aa43-2a0e017e7542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:19.856209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e588d4e-dfb3-4c1d-aa43-2a0e017e7542 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.856478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e588d4e-dfb3-4c1d-aa43-2a0e017e7542 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.856746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e588d4e-dfb3-4c1d-aa43-2a0e017e7542 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.857381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e588d4e-dfb3-4c1d-aa43-2a0e017e7542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:19.858985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e588d4e-dfb3-4c1d-aa43-2a0e017e7542 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:19.869475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 983/1977] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:19.879881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f6fb7-a5ac-4229-b4f1-fa1d926f894b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:19.890405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f6fb7-a5ac-4229-b4f1-fa1d926f894b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:19.895327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01390573-1341-4673-bfb4-3b722b9cc27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:19.895876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390573-1341-4673-bfb4-3b722b9cc27c admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:19.896150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390573-1341-4673-bfb4-3b722b9cc27c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:19.896416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390573-1341-4673-bfb4-3b722b9cc27c admin admin] PATH_INFO: `/v3/projects/d0eabad47bab407bb7aaa352dcbf4140/users/544a403dc3e3405395d47be3c79f3de2/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:19.914110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01390573-1341-4673-bfb4-3b722b9cc27c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0eabad47bab407bb7aaa352dcbf4140, user_id=544a403dc3e3405395d47be3c79f3de2, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:19.916842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01390573-1341-4673-bfb4-3b722b9cc27c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:20.031207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 995/1978] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:19 2022] PUT /identity/v3/projects/d0eabad47bab407bb7aaa352dcbf4140/users/544a403dc3e3405395d47be3c79f3de2/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:20.043502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc02aea-8da4-40e9-b883-e7d4e7709391 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:20.043787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc02aea-8da4-40e9-b883-e7d4e7709391 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:20.044237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc02aea-8da4-40e9-b883-e7d4e7709391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:20.074688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7dc02aea-8da4-40e9-b883-e7d4e7709391 None None] MFA Rules not processed for user `544a403dc3e3405395d47be3c79f3de2`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:20.074958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7dc02aea-8da4-40e9-b883-e7d4e7709391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:20.139018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 984/1979] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:20.143615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01390573-1341-4673-bfb4-3b722b9cc27c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:20.207684 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01390573-1341-4673-bfb4-3b722b9cc27c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:20.212434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbc0b5-6137-44ff-aebc-afe7a083d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:20.213079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbc0b5-6137-44ff-aebc-afe7a083d26a admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:20.213359 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbc0b5-6137-44ff-aebc-afe7a083d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:20.213631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbc0b5-6137-44ff-aebc-afe7a083d26a admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:20.214255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dbc0b5-6137-44ff-aebc-afe7a083d26a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:20.217288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dbc0b5-6137-44ff-aebc-afe7a083d26a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:20.376347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 996/1980] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:20 2022] POST /identity/v3/projects => generated 355 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:58:20.380554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc02aea-8da4-40e9-b883-e7d4e7709391 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:20.392582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc02aea-8da4-40e9-b883-e7d4e7709391 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:20.456532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a09041-6d68-42b6-bc7e-8db9c5140c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:20.457152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a09041-6d68-42b6-bc7e-8db9c5140c38 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:20.457444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a09041-6d68-42b6-bc7e-8db9c5140c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:20.457738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a09041-6d68-42b6-bc7e-8db9c5140c38 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:20.458301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a09041-6d68-42b6-bc7e-8db9c5140c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:20.460738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a09041-6d68-42b6-bc7e-8db9c5140c38 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:20.500426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:20.598725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 985/1981] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:20 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:58:20.601687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbc0b5-6137-44ff-aebc-afe7a083d26a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:20.611951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbc0b5-6137-44ff-aebc-afe7a083d26a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:20.616511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd94f35-8160-42fd-9121-5343ba815b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:20.617100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd94f35-8160-42fd-9121-5343ba815b92 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:20.617415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd94f35-8160-42fd-9121-5343ba815b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:20.617682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd94f35-8160-42fd-9121-5343ba815b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:20.618347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd94f35-8160-42fd-9121-5343ba815b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:20.619957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd94f35-8160-42fd-9121-5343ba815b92 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:20.631157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 997/1982] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:20.635248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a09041-6d68-42b6-bc7e-8db9c5140c38 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:20.646025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a09041-6d68-42b6-bc7e-8db9c5140c38 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:20.650722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b029522-1161-4714-bf73-7dbb92aa6cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:20.657367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b029522-1161-4714-bf73-7dbb92aa6cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:20.657367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b029522-1161-4714-bf73-7dbb92aa6cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:20.657367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b029522-1161-4714-bf73-7dbb92aa6cb7 admin admin] PATH_INFO: `/v3/projects/3f6ab7a0203c49a9872135b1a02def6f/users/446e3960a272424480e55b34563fe0ab/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:20.668883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b029522-1161-4714-bf73-7dbb92aa6cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6ab7a0203c49a9872135b1a02def6f, user_id=446e3960a272424480e55b34563fe0ab, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:20.671669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b029522-1161-4714-bf73-7dbb92aa6cb7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:20.711882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:20.832135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 986/1983] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:20 2022] PUT /identity/v3/projects/3f6ab7a0203c49a9872135b1a02def6f/users/446e3960a272424480e55b34563fe0ab/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:58:20.836114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd94f35-8160-42fd-9121-5343ba815b92 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:20.846813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd94f35-8160-42fd-9121-5343ba815b92 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:20.851580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:20.903047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90d873-540f-47c6-8f0c-041d394227a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:20.903686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90d873-540f-47c6-8f0c-041d394227a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:20.903968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90d873-540f-47c6-8f0c-041d394227a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:20.904238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90d873-540f-47c6-8f0c-041d394227a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:20.904896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90d873-540f-47c6-8f0c-041d394227a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:20.906508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90d873-540f-47c6-8f0c-041d394227a2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:20.916349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 998/1984] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:20.925255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b029522-1161-4714-bf73-7dbb92aa6cb7 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:20.931392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b029522-1161-4714-bf73-7dbb92aa6cb7 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:20.936388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:20.937042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:20.937319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:20.937591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] PATH_INFO: `/v3/projects/3f6ab7a0203c49a9872135b1a02def6f/users/446e3960a272424480e55b34563fe0ab/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:20.939566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6ab7a0203c49a9872135b1a02def6f, user_id=446e3960a272424480e55b34563fe0ab, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:20.942237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:20.953944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 987/1985] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:20 2022] PUT /identity/v3/projects/3f6ab7a0203c49a9872135b1a02def6f/users/446e3960a272424480e55b34563fe0ab/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:20.965230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3987e2-0ae4-481c-ac5b-0182fbd0aee9 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:20.965496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3987e2-0ae4-481c-ac5b-0182fbd0aee9 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:20.965777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3987e2-0ae4-481c-ac5b-0182fbd0aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:20.994571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5a3987e2-0ae4-481c-ac5b-0182fbd0aee9 None None] MFA Rules not processed for user `446e3960a272424480e55b34563fe0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:20.996583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5a3987e2-0ae4-481c-ac5b-0182fbd0aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:21.065699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 999/1986] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:21.078103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:21.079881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:21.139818 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30bfa-b980-46ce-a75d-c3a97fc2b137 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:21.143417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860b2b2-681d-491e-954b-68869841e6da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:21.144029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860b2b2-681d-491e-954b-68869841e6da service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:21.144307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860b2b2-681d-491e-954b-68869841e6da service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:21.144804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860b2b2-681d-491e-954b-68869841e6da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:21.156126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860b2b2-681d-491e-954b-68869841e6da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:21.157574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860b2b2-681d-491e-954b-68869841e6da service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:21.167415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 988/1987] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:21.474480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3987e2-0ae4-481c-ac5b-0182fbd0aee9 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:21.484436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3987e2-0ae4-481c-ac5b-0182fbd0aee9 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:21.488502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:21.488945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:21.489328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:21.489677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:21.581937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:21.583640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:21.594851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1000/1988] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:22.408616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904500-207f-47bc-b943-1ef83f103cf9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:22.409165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904500-207f-47bc-b943-1ef83f103cf9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:22.409165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904500-207f-47bc-b943-1ef83f103cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:22.430773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8b904500-207f-47bc-b943-1ef83f103cf9 None None] MFA Rules not processed for user `a0ecfd3d64ba4fef820a31b1522125ff`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:22.432066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8b904500-207f-47bc-b943-1ef83f103cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:22.519825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 989/1989] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:27.982441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:27.983945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:28.018988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:28.052255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a51483-56e1-4e5c-a3b9-8b382b740ec9 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:28.056862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:28.057726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.057843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.058621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] PATH_INFO: `/v3/users/88b66441b1434f94ba277ab46df963ac` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.060101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b66441b1434f94ba277ab46df963ac)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:28.064014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:28.309416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b66441b1434f94ba277ab46df963ac'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.310075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b66441b1434f94ba277ab46df963ac'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.322506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b66441b1434f94ba277ab46df963ac'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.414504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b66441b1434f94ba277ab46df963ac'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.422549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b66441b1434f94ba277ab46df963ac'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:28.429445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1001/1990] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:27 2022] DELETE /identity/v3/users/88b66441b1434f94ba277ab46df963ac => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:28.434958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a2b4-51af-46c9-a728-123cbbbd6195 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.435480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a2b4-51af-46c9-a728-123cbbbd6195 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.435572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a2b4-51af-46c9-a728-123cbbbd6195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.454362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5064a2b4-51af-46c9-a728-123cbbbd6195 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:28.455650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5064a2b4-51af-46c9-a728-123cbbbd6195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:28.464097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:28.516488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 990/1991] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:28.526212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:28.527824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:28.590093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d61c0-09f4-41f5-95f6-ceac4bf97567 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:28.593783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ca43e-a1ea-41b7-a026-0a1222f5e06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:28.594400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca43e-a1ea-41b7-a026-0a1222f5e06c service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:28.594675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca43e-a1ea-41b7-a026-0a1222f5e06c service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:28.594972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca43e-a1ea-41b7-a026-0a1222f5e06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:28.605477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ca43e-a1ea-41b7-a026-0a1222f5e06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:28.606862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ca43e-a1ea-41b7-a026-0a1222f5e06c service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:28.617016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1002/1992] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:29.179370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064a2b4-51af-46c9-a728-123cbbbd6195 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:29.181095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5064a2b4-51af-46c9-a728-123cbbbd6195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:29.202152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064a2b4-51af-46c9-a728-123cbbbd6195 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.212313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.213035 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.213971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.215366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] PATH_INFO: `/v3/projects/377bc47cac8e4cd6821ecdb74755c741` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.217027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] RBAC: Authorizing `identity:delete_project(project_id=377bc47cac8e4cd6821ecdb74755c741)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.218973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:29.359663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '377bc47cac8e4cd6821ecdb74755c741', 'request_id': 'req-88dead4d-fc89-43ad-8f54-cc8666ec167f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:29.369279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '377bc47cac8e4cd6821ecdb74755c741', 'request_id': 'req-88dead4d-fc89-43ad-8f54-cc8666ec167f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:29.450833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '377bc47cac8e4cd6821ecdb74755c741', 'request_id': 'req-88dead4d-fc89-43ad-8f54-cc8666ec167f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:29.467317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] The token cache is being invalidate because project 377bc47cac8e4cd6821ecdb74755c741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:29.467727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:29.468522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-88dead4d-fc89-43ad-8f54-cc8666ec167f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:29.471002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 991/1993] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:29 2022] DELETE /identity/v3/projects/377bc47cac8e4cd6821ecdb74755c741 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:29.475850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823275dd-94af-48ec-9cd9-0f84da397503 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.476402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823275dd-94af-48ec-9cd9-0f84da397503 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.476674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823275dd-94af-48ec-9cd9-0f84da397503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.510859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-823275dd-94af-48ec-9cd9-0f84da397503 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:29.513125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-823275dd-94af-48ec-9cd9-0f84da397503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:29.578885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:29.594884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1003/1994] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:29.603493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811c04-fed4-458a-ab2a-c37bec52ca91 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.603776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811c04-fed4-458a-ab2a-c37bec52ca91 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.604052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811c04-fed4-458a-ab2a-c37bec52ca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.625945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-cf811c04-fed4-458a-ab2a-c37bec52ca91 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:29.627206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-cf811c04-fed4-458a-ab2a-c37bec52ca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:29.635651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 992/1995] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:29.640536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823275dd-94af-48ec-9cd9-0f84da397503 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:29.650559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823275dd-94af-48ec-9cd9-0f84da397503 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.655440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.656049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.656327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.656597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.657266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:58:29.657548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.659275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:29.672415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1004/1996] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:58:29.676687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811c04-fed4-458a-ab2a-c37bec52ca91 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:29.687335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811c04-fed4-458a-ab2a-c37bec52ca91 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.692156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26181f1-74e8-43b6-9e0b-59b526dda171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.692838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26181f1-74e8-43b6-9e0b-59b526dda171 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.693183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26181f1-74e8-43b6-9e0b-59b526dda171 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.693519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26181f1-74e8-43b6-9e0b-59b526dda171 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.694253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26181f1-74e8-43b6-9e0b-59b526dda171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.696253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26181f1-74e8-43b6-9e0b-59b526dda171 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:29.875732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 993/1997] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:29 2022] POST /identity/v3/projects => generated 369 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:29.877210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:29.891848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c8ab5-3f4d-4c56-b9d4-40f0211367d6 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:29.972996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d6f94-c133-452a-9833-54804f379b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:29.973617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6f94-c133-452a-9833-54804f379b73 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:29.973895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6f94-c133-452a-9833-54804f379b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:29.974194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6f94-c133-452a-9833-54804f379b73 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:29.974807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d6f94-c133-452a-9833-54804f379b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:29.976786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d6f94-c133-452a-9833-54804f379b73 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.258973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1005/1998] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:29 2022] POST /identity/v3/users => generated 285 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:30.264100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26181f1-74e8-43b6-9e0b-59b526dda171 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:30.279869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26181f1-74e8-43b6-9e0b-59b526dda171 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:30.296854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb311252-ed63-4f9a-be71-75c102506668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:30.297507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb311252-ed63-4f9a-be71-75c102506668 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.298016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb311252-ed63-4f9a-be71-75c102506668 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.298288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb311252-ed63-4f9a-be71-75c102506668 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.299010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb311252-ed63-4f9a-be71-75c102506668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:30.300647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb311252-ed63-4f9a-be71-75c102506668 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.314319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 994/1999] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:30.317959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d6f94-c133-452a-9833-54804f379b73 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:30.337585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d6f94-c133-452a-9833-54804f379b73 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:30.349027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4272-df17-44d2-92ea-1a5e10b0b8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:30.350119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4272-df17-44d2-92ea-1a5e10b0b8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.351052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4272-df17-44d2-92ea-1a5e10b0b8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.351490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4272-df17-44d2-92ea-1a5e10b0b8a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.352242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa4272-df17-44d2-92ea-1a5e10b0b8a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:30.356627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa4272-df17-44d2-92ea-1a5e10b0b8a7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.387562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1006/2000] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:30.389120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb311252-ed63-4f9a-be71-75c102506668 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:30.406366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb311252-ed63-4f9a-be71-75c102506668 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:30.414407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:30.415205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.415679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.415985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] PATH_INFO: `/v3/projects/5efcc0d6257448329354e6a23d60f433/users/50f3ad272b0e4d7db63df288c3977e47/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.435572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efcc0d6257448329354e6a23d60f433, user_id=50f3ad272b0e4d7db63df288c3977e47, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:30.439066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.568418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 995/2001] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:30 2022] PUT /identity/v3/projects/5efcc0d6257448329354e6a23d60f433/users/50f3ad272b0e4d7db63df288c3977e47/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:30.579482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c2a0-10b7-4d28-8c2d-28bb2299465b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.579893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c2a0-10b7-4d28-8c2d-28bb2299465b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.579987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756c2a0-10b7-4d28-8c2d-28bb2299465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.613332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d756c2a0-10b7-4d28-8c2d-28bb2299465b None None] MFA Rules not processed for user `50f3ad272b0e4d7db63df288c3977e47`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:30.614669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d756c2a0-10b7-4d28-8c2d-28bb2299465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:30.688201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1007/2002] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:30.699931 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:30.701873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:30.750229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:30.794248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a347e5a-cabe-486c-bdbe-45ad1064837b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:30.794624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23ddf8f600454da7ae9b715cd4cbd186', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:30.795266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:30.795549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:30.795826 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:30.805558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:30.807036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:30.817301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 996/2003] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:32.156035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699406d-8b9c-4a1b-a2c3-50d41f4a269f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:32.156622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699406d-8b9c-4a1b-a2c3-50d41f4a269f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:32.156622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699406d-8b9c-4a1b-a2c3-50d41f4a269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:32.176328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1699406d-8b9c-4a1b-a2c3-50d41f4a269f None None] MFA Rules not processed for user `abe845d55b4b4ac898556dbdc264c086`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:32.177617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1699406d-8b9c-4a1b-a2c3-50d41f4a269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:32.236515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:32.256361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1008/2004] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:41.845993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=20 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:43.117238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:44.010718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:44.012893 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:44.081945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec6aff-e2aa-4ae4-99f5-d12917927da6 service glance] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:44.087263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:44.088112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:44.088391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:44.088690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] PATH_INFO: `/v3/users/544a403dc3e3405395d47be3c79f3de2` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:44.089864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=544a403dc3e3405395d47be3c79f3de2)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:44.091868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:44.315392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544a403dc3e3405395d47be3c79f3de2'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:44.316649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544a403dc3e3405395d47be3c79f3de2'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:44.336389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544a403dc3e3405395d47be3c79f3de2'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:44.338854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544a403dc3e3405395d47be3c79f3de2'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:44.455008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544a403dc3e3405395d47be3c79f3de2'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:44.471133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 997/2005] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:44 2022] DELETE /identity/v3/users/544a403dc3e3405395d47be3c79f3de2 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:44.475718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994456-0b2d-4fc5-bfdf-f0d684b26444 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:44.482897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994456-0b2d-4fc5-bfdf-f0d684b26444 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:44.482897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee994456-0b2d-4fc5-bfdf-f0d684b26444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:44.497377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ee994456-0b2d-4fc5-bfdf-f0d684b26444 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:44.498592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ee994456-0b2d-4fc5-bfdf-f0d684b26444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:44.567107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1009/2006] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:44.578253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:44.579985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:44.643881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0495014-befd-4960-b72c-b512324658b8 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:44.648170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a61590-d2f9-4157-b4bd-7cc5e5ef41f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:44.648795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61590-d2f9-4157-b4bd-7cc5e5ef41f7 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:44.649072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61590-d2f9-4157-b4bd-7cc5e5ef41f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:44.649366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61590-d2f9-4157-b4bd-7cc5e5ef41f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:44.665179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a61590-d2f9-4157-b4bd-7cc5e5ef41f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:44.667867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a61590-d2f9-4157-b4bd-7cc5e5ef41f7 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:44.679346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 998/2007] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:45.315773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994456-0b2d-4fc5-bfdf-f0d684b26444 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:45.318149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ee994456-0b2d-4fc5-bfdf-f0d684b26444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:45.352143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee994456-0b2d-4fc5-bfdf-f0d684b26444 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:45.359999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:45.360645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:45.361281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:45.361416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] PATH_INFO: `/v3/projects/d0eabad47bab407bb7aaa352dcbf4140` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:45.364431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0eabad47bab407bb7aaa352dcbf4140)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:45.369190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:45.526331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0eabad47bab407bb7aaa352dcbf4140', 'request_id': 'req-981db38f-0349-41c8-b8cf-3262171aab01'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:45.532936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0eabad47bab407bb7aaa352dcbf4140', 'request_id': 'req-981db38f-0349-41c8-b8cf-3262171aab01'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:45.611106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0eabad47bab407bb7aaa352dcbf4140', 'request_id': 'req-981db38f-0349-41c8-b8cf-3262171aab01'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:45.624664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] The token cache is being invalidate because project d0eabad47bab407bb7aaa352dcbf4140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:45.624956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:45.625614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:45.627274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1010/2008] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:45 2022] DELETE /identity/v3/projects/d0eabad47bab407bb7aaa352dcbf4140 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:45.631450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a61590-d2f9-4157-b4bd-7cc5e5ef41f7 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:45.641798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a61590-d2f9-4157-b4bd-7cc5e5ef41f7 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:45.664961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:45.712582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:45.713222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:45.713496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:45.713826 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] PATH_INFO: `/v3/users/446e3960a272424480e55b34563fe0ab` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:45.715814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=446e3960a272424480e55b34563fe0ab)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:45.717946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:45.917999 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '446e3960a272424480e55b34563fe0ab'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:45.918721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '446e3960a272424480e55b34563fe0ab'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:45.931964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '446e3960a272424480e55b34563fe0ab'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:45.933426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '446e3960a272424480e55b34563fe0ab'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:46.034240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a4553c90-1053-4855-84c6-740437c2a4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '446e3960a272424480e55b34563fe0ab'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:46.048195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 999/2009] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:45 2022] DELETE /identity/v3/users/446e3960a272424480e55b34563fe0ab => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:46.527660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:46.529360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:46.544755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981db38f-0349-41c8-b8cf-3262171aab01 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:46.614540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:46.615204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:46.615481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:46.615749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] PATH_INFO: `/v3/projects/3f6ab7a0203c49a9872135b1a02def6f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:46.617180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f6ab7a0203c49a9872135b1a02def6f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:46.619135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:46.748292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:46.768572 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f6ab7a0203c49a9872135b1a02def6f', 'request_id': 'req-a0861441-8885-4cbb-8762-11594b0fcfad'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:46.774363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f6ab7a0203c49a9872135b1a02def6f', 'request_id': 'req-a0861441-8885-4cbb-8762-11594b0fcfad'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:46.862856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f6ab7a0203c49a9872135b1a02def6f', 'request_id': 'req-a0861441-8885-4cbb-8762-11594b0fcfad'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:46.876391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] The token cache is being invalidate because project 3f6ab7a0203c49a9872135b1a02def6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:46.876690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:46.877347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a0861441-8885-4cbb-8762-11594b0fcfad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:46.878840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1011/2010] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:46 2022] DELETE /identity/v3/projects/3f6ab7a0203c49a9872135b1a02def6f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:46.883748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4070fc-ff20-4071-aba5-61283fdd737c None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:46.884159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4070fc-ff20-4071-aba5-61283fdd737c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:46.884336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4070fc-ff20-4071-aba5-61283fdd737c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:46.904706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ce4070fc-ff20-4071-aba5-61283fdd737c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:46.909355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ce4070fc-ff20-4071-aba5-61283fdd737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:47.019580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1000/2011] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:47.024337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccef19-4f9a-4515-a4c2-7a9657ca9767 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.024625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccef19-4f9a-4515-a4c2-7a9657ca9767 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.024898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccef19-4f9a-4515-a4c2-7a9657ca9767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.055257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-35ccef19-4f9a-4515-a4c2-7a9657ca9767 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:47.057100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-35ccef19-4f9a-4515-a4c2-7a9657ca9767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:47.075161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1012/2012] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:47.079500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4070fc-ff20-4071-aba5-61283fdd737c None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:47.090741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4070fc-ff20-4071-aba5-61283fdd737c None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:47.095592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:47.096342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.096693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.097062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.097966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:58:47.098120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:47.099776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:47.122977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1001/2013] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:58:47.123309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccef19-4f9a-4515-a4c2-7a9657ca9767 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:47.130352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccef19-4f9a-4515-a4c2-7a9657ca9767 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:47.136843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b1efe-bfbd-4f70-a4a8-416a58c6efeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:47.137448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b1efe-bfbd-4f70-a4a8-416a58c6efeb admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.137718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b1efe-bfbd-4f70-a4a8-416a58c6efeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.137980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b1efe-bfbd-4f70-a4a8-416a58c6efeb admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.138623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b1efe-bfbd-4f70-a4a8-416a58c6efeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:47.140541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b1efe-bfbd-4f70-a4a8-416a58c6efeb admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:47.230603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1013/2014] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:47 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:47.234803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:47.245676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac46e1-135c-45c3-b450-a350a1249137 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:47.271331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:47.304121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a57a7-56f7-4cfb-b8e5-212fae035dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:47.304739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a57a7-56f7-4cfb-b8e5-212fae035dbd admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.304996 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a57a7-56f7-4cfb-b8e5-212fae035dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.305257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a57a7-56f7-4cfb-b8e5-212fae035dbd admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.305881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a57a7-56f7-4cfb-b8e5-212fae035dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:47.308913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a57a7-56f7-4cfb-b8e5-212fae035dbd admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:47.403578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1002/2015] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:47 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:47.407781 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b1efe-bfbd-4f70-a4a8-416a58c6efeb admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:47.421476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b1efe-bfbd-4f70-a4a8-416a58c6efeb admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:47.427976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7daae06-f303-4775-b28a-9e7a7e21188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:47.428568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7daae06-f303-4775-b28a-9e7a7e21188a admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.428865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7daae06-f303-4775-b28a-9e7a7e21188a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.429136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7daae06-f303-4775-b28a-9e7a7e21188a admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.429786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7daae06-f303-4775-b28a-9e7a7e21188a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:47.431428 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7daae06-f303-4775-b28a-9e7a7e21188a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:47.446073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1014/2016] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:58:47.448595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a57a7-56f7-4cfb-b8e5-212fae035dbd admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:47.461501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a57a7-56f7-4cfb-b8e5-212fae035dbd admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:47.469236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61137f8-ad79-4bf9-9c67-66b0fd956e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:47.469850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61137f8-ad79-4bf9-9c67-66b0fd956e3b admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.470124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61137f8-ad79-4bf9-9c67-66b0fd956e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.470443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61137f8-ad79-4bf9-9c67-66b0fd956e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.471130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61137f8-ad79-4bf9-9c67-66b0fd956e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:47.473232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61137f8-ad79-4bf9-9c67-66b0fd956e3b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:47.485626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1003/2017] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:47.489786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7daae06-f303-4775-b28a-9e7a7e21188a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:47.501080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7daae06-f303-4775-b28a-9e7a7e21188a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:47.505737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe4e9-982e-473a-8329-d9237255a496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:47.506479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe4e9-982e-473a-8329-d9237255a496 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.506766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe4e9-982e-473a-8329-d9237255a496 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.507063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe4e9-982e-473a-8329-d9237255a496 admin admin] PATH_INFO: `/v3/projects/556abb9db8f84cc7b86058d431abf693/users/11b2437549ec4a5da7d6bae86bbca91e/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.526712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfe4e9-982e-473a-8329-d9237255a496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556abb9db8f84cc7b86058d431abf693, user_id=11b2437549ec4a5da7d6bae86bbca91e, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:47.529486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfe4e9-982e-473a-8329-d9237255a496 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:47.599972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1015/2018] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:47 2022] PUT /identity/v3/projects/556abb9db8f84cc7b86058d431abf693/users/11b2437549ec4a5da7d6bae86bbca91e/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:47.613718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46135c9-ae84-4638-aae1-1a379199c3ba None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.614114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46135c9-ae84-4638-aae1-1a379199c3ba None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.614483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46135c9-ae84-4638-aae1-1a379199c3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.649771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d46135c9-ae84-4638-aae1-1a379199c3ba None None] MFA Rules not processed for user `11b2437549ec4a5da7d6bae86bbca91e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:47.651624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d46135c9-ae84-4638-aae1-1a379199c3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:47.725405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1004/2019] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:47.726545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe4e9-982e-473a-8329-d9237255a496 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:47.832054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe4e9-982e-473a-8329-d9237255a496 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:47.837193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6f6a-52b1-4f91-9fe7-c71507d0058a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:47.837841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6f6a-52b1-4f91-9fe7-c71507d0058a admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:47.838121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6f6a-52b1-4f91-9fe7-c71507d0058a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:47.838599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6f6a-52b1-4f91-9fe7-c71507d0058a admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:47.839207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6f6a-52b1-4f91-9fe7-c71507d0058a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:47.841141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6f6a-52b1-4f91-9fe7-c71507d0058a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:47.949943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1016/2020] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:47 2022] POST /identity/v3/projects => generated 363 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:47.954146 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46135c9-ae84-4638-aae1-1a379199c3ba None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:47.965369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46135c9-ae84-4638-aae1-1a379199c3ba None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:48.034281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462e488-d5bb-4364-baef-0b261c5f21ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:48.035469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462e488-d5bb-4364-baef-0b261c5f21ee admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:48.035748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462e488-d5bb-4364-baef-0b261c5f21ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:48.036020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462e488-d5bb-4364-baef-0b261c5f21ee admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:48.036663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462e488-d5bb-4364-baef-0b261c5f21ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:48.039032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462e488-d5bb-4364-baef-0b261c5f21ee admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:48.215063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1005/2021] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:47 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:48.216187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6f6a-52b1-4f91-9fe7-c71507d0058a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:48.226697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6f6a-52b1-4f91-9fe7-c71507d0058a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:48.231340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19521a-d3fb-4e6c-ac77-cad65f568a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:48.232625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19521a-d3fb-4e6c-ac77-cad65f568a9d admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:48.232625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19521a-d3fb-4e6c-ac77-cad65f568a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:48.232625 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19521a-d3fb-4e6c-ac77-cad65f568a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:48.233175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19521a-d3fb-4e6c-ac77-cad65f568a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:48.234764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19521a-d3fb-4e6c-ac77-cad65f568a9d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:48.246217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1017/2022] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:48.250648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462e488-d5bb-4364-baef-0b261c5f21ee admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:48.263958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462e488-d5bb-4364-baef-0b261c5f21ee admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:48.278430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488bd98-fea3-41fd-a6de-9529c9307d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:48.279133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488bd98-fea3-41fd-a6de-9529c9307d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:48.279407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488bd98-fea3-41fd-a6de-9529c9307d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:48.280334 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488bd98-fea3-41fd-a6de-9529c9307d48 admin admin] PATH_INFO: `/v3/projects/f85d475f10a04971b0844f64ca48149b/users/da9bb57593834b6e90faf2a89a3fd513/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:48.301858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8488bd98-fea3-41fd-a6de-9529c9307d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85d475f10a04971b0844f64ca48149b, user_id=da9bb57593834b6e90faf2a89a3fd513, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:48.304547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8488bd98-fea3-41fd-a6de-9529c9307d48 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:48.426960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1006/2023] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:48 2022] PUT /identity/v3/projects/f85d475f10a04971b0844f64ca48149b/users/da9bb57593834b6e90faf2a89a3fd513/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:48.430831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19521a-d3fb-4e6c-ac77-cad65f568a9d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:48.442150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19521a-d3fb-4e6c-ac77-cad65f568a9d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:48.463975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:48.501627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f861e25-a8f1-486a-8820-9e01f304e0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:48.502162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861e25-a8f1-486a-8820-9e01f304e0fe admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:48.502439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861e25-a8f1-486a-8820-9e01f304e0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:48.502708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861e25-a8f1-486a-8820-9e01f304e0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:48.503381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f861e25-a8f1-486a-8820-9e01f304e0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:48.505027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f861e25-a8f1-486a-8820-9e01f304e0fe admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:48.515798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1018/2024] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:58:48.519697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488bd98-fea3-41fd-a6de-9529c9307d48 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:48.525195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:48.530678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488bd98-fea3-41fd-a6de-9529c9307d48 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:48.535354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:48.536005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:48.536283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:48.536551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] PATH_INFO: `/v3/projects/f85d475f10a04971b0844f64ca48149b/users/da9bb57593834b6e90faf2a89a3fd513/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:48.538456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85d475f10a04971b0844f64ca48149b, user_id=da9bb57593834b6e90faf2a89a3fd513, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:48.541192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:48.552926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1007/2025] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:48 2022] PUT /identity/v3/projects/f85d475f10a04971b0844f64ca48149b/users/da9bb57593834b6e90faf2a89a3fd513/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:48.565913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0dc9d-f0a6-447c-8c40-10a37abb8f19 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:48.566488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0dc9d-f0a6-447c-8c40-10a37abb8f19 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:48.566488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0dc9d-f0a6-447c-8c40-10a37abb8f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:48.595871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-58b0dc9d-f0a6-447c-8c40-10a37abb8f19 None None] MFA Rules not processed for user `da9bb57593834b6e90faf2a89a3fd513`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:48.597088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-58b0dc9d-f0a6-447c-8c40-10a37abb8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:48.677819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1019/2026] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:48.689917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:48.692836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:48.755248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedc10-e16c-4744-be0f-39404fc76da8 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:48.759046 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d049f1-afb8-4e51-89ad-488041070f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:48.759654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d049f1-afb8-4e51-89ad-488041070f69 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:48.760004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d049f1-afb8-4e51-89ad-488041070f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:48.760278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d049f1-afb8-4e51-89ad-488041070f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:48.775307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d049f1-afb8-4e51-89ad-488041070f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:48.779050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d049f1-afb8-4e51-89ad-488041070f69 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:48.789761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1008/2027] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:49.061198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0dc9d-f0a6-447c-8c40-10a37abb8f19 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:49.062686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-58b0dc9d-f0a6-447c-8c40-10a37abb8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:49.074861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0dc9d-f0a6-447c-8c40-10a37abb8f19 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:49.084078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae5abf-6d00-44c7-898a-b557abf7992f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:49.084692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae5abf-6d00-44c7-898a-b557abf7992f service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:49.084975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae5abf-6d00-44c7-898a-b557abf7992f service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:49.085251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae5abf-6d00-44c7-898a-b557abf7992f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:49.146734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae5abf-6d00-44c7-898a-b557abf7992f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:49.148348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae5abf-6d00-44c7-898a-b557abf7992f service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:49.157990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1020/2028] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:52.965138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d049f1-afb8-4e51-89ad-488041070f69 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:53.042455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d049f1-afb8-4e51-89ad-488041070f69 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:53.047722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:53.048450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:53.048733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:53.049011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] PATH_INFO: `/v3/users/11b2437549ec4a5da7d6bae86bbca91e` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:53.050182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b2437549ec4a5da7d6bae86bbca91e)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:53.052174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:53.404690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b2437549ec4a5da7d6bae86bbca91e'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:53.405563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b2437549ec4a5da7d6bae86bbca91e'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:53.422059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b2437549ec4a5da7d6bae86bbca91e'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:53.424703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b2437549ec4a5da7d6bae86bbca91e'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:53.565211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b2437549ec4a5da7d6bae86bbca91e'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:53.584610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1009/2029] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:52 2022] DELETE /identity/v3/users/11b2437549ec4a5da7d6bae86bbca91e => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:53.589380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e41f3-a6ac-47ed-856c-e86c9d0dcabc None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:53.589890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e41f3-a6ac-47ed-856c-e86c9d0dcabc None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:53.590003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e41f3-a6ac-47ed-856c-e86c9d0dcabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:53.608853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-590e41f3-a6ac-47ed-856c-e86c9d0dcabc None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:53.610031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-590e41f3-a6ac-47ed-856c-e86c9d0dcabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:53.652884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:53.672126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1021/2030] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:53.681935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:53.683619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:53.696913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:53.746678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce94e3-a7ee-47e6-a22d-7d4fe1b36651 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:53.750435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8199ac-8e11-44ab-a2ef-fe2702cb2c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:53.751089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8199ac-8e11-44ab-a2ef-fe2702cb2c99 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:53.751365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8199ac-8e11-44ab-a2ef-fe2702cb2c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:53.751633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8199ac-8e11-44ab-a2ef-fe2702cb2c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:53.761806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8199ac-8e11-44ab-a2ef-fe2702cb2c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:53.763276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8199ac-8e11-44ab-a2ef-fe2702cb2c99 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:53.773096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1010/2031] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:58:54.212521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e41f3-a6ac-47ed-856c-e86c9d0dcabc None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:54.214808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-590e41f3-a6ac-47ed-856c-e86c9d0dcabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:54.233361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e41f3-a6ac-47ed-856c-e86c9d0dcabc None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:54.238954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:54.239727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:54.240007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:54.240275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] PATH_INFO: `/v3/projects/556abb9db8f84cc7b86058d431abf693` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:54.241478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=556abb9db8f84cc7b86058d431abf693)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:54.245563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:54.370848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556abb9db8f84cc7b86058d431abf693', 'request_id': 'req-0fef69b1-584e-483c-9932-40e7de287416'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.377112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556abb9db8f84cc7b86058d431abf693', 'request_id': 'req-0fef69b1-584e-483c-9932-40e7de287416'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.496563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556abb9db8f84cc7b86058d431abf693', 'request_id': 'req-0fef69b1-584e-483c-9932-40e7de287416'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.508333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] The token cache is being invalidate because project 556abb9db8f84cc7b86058d431abf693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:54.508615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.509296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.511587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1022/2032] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:54 2022] DELETE /identity/v3/projects/556abb9db8f84cc7b86058d431abf693 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:54.515700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8199ac-8e11-44ab-a2ef-fe2702cb2c99 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:54.525523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8199ac-8e11-44ab-a2ef-fe2702cb2c99 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:54.587633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:54.588270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:54.588606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:54.588912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] PATH_INFO: `/v3/users/da9bb57593834b6e90faf2a89a3fd513` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:54.590653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9bb57593834b6e90faf2a89a3fd513)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:54.592608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:54.772912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9bb57593834b6e90faf2a89a3fd513'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.773564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9bb57593834b6e90faf2a89a3fd513'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.785743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9bb57593834b6e90faf2a89a3fd513'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.786429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9bb57593834b6e90faf2a89a3fd513'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.856672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a19857f6-bce2-47f9-802d-f66217c274fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9bb57593834b6e90faf2a89a3fd513'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:54.869475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1011/2033] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:58:54 2022] DELETE /identity/v3/users/da9bb57593834b6e90faf2a89a3fd513 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:55.414892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:55.418110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:55.460457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef69b1-584e-483c-9932-40e7de287416 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:55.543644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:55.544671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:55.545136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:55.545605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] PATH_INFO: `/v3/projects/f85d475f10a04971b0844f64ca48149b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:55.547025 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85d475f10a04971b0844f64ca48149b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:55.549219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:55.680063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85d475f10a04971b0844f64ca48149b', 'request_id': 'req-28e3b53e-d543-4750-ba26-c222d1951fa9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:55.688312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85d475f10a04971b0844f64ca48149b', 'request_id': 'req-28e3b53e-d543-4750-ba26-c222d1951fa9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:55.799461 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85d475f10a04971b0844f64ca48149b', 'request_id': 'req-28e3b53e-d543-4750-ba26-c222d1951fa9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:55.808368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:55.813638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] The token cache is being invalidate because project f85d475f10a04971b0844f64ca48149b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:58:55.813934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:55.814674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-28e3b53e-d543-4750-ba26-c222d1951fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:58:55.816664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1023/2034] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:58:55 2022] DELETE /identity/v3/projects/f85d475f10a04971b0844f64ca48149b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:55.821580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65fc32-5f3d-4662-a671-26066f9f657e None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:55.822517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65fc32-5f3d-4662-a671-26066f9f657e None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:55.822915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65fc32-5f3d-4662-a671-26066f9f657e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:55.844847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3a65fc32-5f3d-4662-a671-26066f9f657e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:55.846835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3a65fc32-5f3d-4662-a671-26066f9f657e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:55.883314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:55.914708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1012/2035] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:58:55.924506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ded3-ed3c-4073-aa3d-ed4893f3c719 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:55.924874 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ded3-ed3c-4073-aa3d-ed4893f3c719 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:55.925255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5ded3-ed3c-4073-aa3d-ed4893f3c719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:55.947036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-acc5ded3-ed3c-4073-aa3d-ed4893f3c719 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:55.948568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-acc5ded3-ed3c-4073-aa3d-ed4893f3c719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:55.957607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1024/2036] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:55.961942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65fc32-5f3d-4662-a671-26066f9f657e None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:55.972364 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65fc32-5f3d-4662-a671-26066f9f657e None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:55.977442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:55.978059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:55.978365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:55.978682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:55.980003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:58:55.980286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:55.982008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:55.996174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1013/2037] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:58:56.000480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5ded3-ed3c-4073-aa3d-ed4893f3c719 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:56.012382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5ded3-ed3c-4073-aa3d-ed4893f3c719 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:56.017282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb61c0d-1232-48b5-a632-4aee3533708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:56.017886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61c0d-1232-48b5-a632-4aee3533708a admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:56.018162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61c0d-1232-48b5-a632-4aee3533708a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:56.018427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61c0d-1232-48b5-a632-4aee3533708a admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:56.019109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb61c0d-1232-48b5-a632-4aee3533708a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:56.020999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb61c0d-1232-48b5-a632-4aee3533708a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:56.150462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1025/2038] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:55 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:56.155340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:56.166263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b383e0a-073d-4fb8-9aa9-9aade5d89a8f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:56.246975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f9937-ec39-412b-99fe-33f876250656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:56.246975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f9937-ec39-412b-99fe-33f876250656 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:56.246975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f9937-ec39-412b-99fe-33f876250656 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:56.246975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f9937-ec39-412b-99fe-33f876250656 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:56.252075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f9937-ec39-412b-99fe-33f876250656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:56.252075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f9937-ec39-412b-99fe-33f876250656 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:56.325246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1014/2039] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:56 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:56.327090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb61c0d-1232-48b5-a632-4aee3533708a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:56.339825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb61c0d-1232-48b5-a632-4aee3533708a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:56.344952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642464d-1008-4a3e-b95c-4377cf9ada74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:56.345823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642464d-1008-4a3e-b95c-4377cf9ada74 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:56.345958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642464d-1008-4a3e-b95c-4377cf9ada74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:56.346136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642464d-1008-4a3e-b95c-4377cf9ada74 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:56.346862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642464d-1008-4a3e-b95c-4377cf9ada74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:56.348652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642464d-1008-4a3e-b95c-4377cf9ada74 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:56.360371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1026/2040] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:56.364212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f9937-ec39-412b-99fe-33f876250656 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:56.374498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f9937-ec39-412b-99fe-33f876250656 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:56.381503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19210cd4-85e5-4727-a316-92e9e5819075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:56.382114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19210cd4-85e5-4727-a316-92e9e5819075 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:56.382389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19210cd4-85e5-4727-a316-92e9e5819075 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:56.382654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19210cd4-85e5-4727-a316-92e9e5819075 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:56.383333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19210cd4-85e5-4727-a316-92e9e5819075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:56.384917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19210cd4-85e5-4727-a316-92e9e5819075 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:56.396710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1015/2041] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:56.401456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642464d-1008-4a3e-b95c-4377cf9ada74 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:56.411923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642464d-1008-4a3e-b95c-4377cf9ada74 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:56.418292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c1177-e3dd-4e9d-874f-d876bf1dde1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:56.418981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1177-e3dd-4e9d-874f-d876bf1dde1b admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:56.419263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1177-e3dd-4e9d-874f-d876bf1dde1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:56.419530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1177-e3dd-4e9d-874f-d876bf1dde1b admin admin] PATH_INFO: `/v3/projects/c0ef6928958e411b9af4fd6cccafed84/users/3bcb8308b527441283b209c605cb5ca9/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:56.435698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c1177-e3dd-4e9d-874f-d876bf1dde1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ef6928958e411b9af4fd6cccafed84, user_id=3bcb8308b527441283b209c605cb5ca9, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:56.438443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c1177-e3dd-4e9d-874f-d876bf1dde1b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:56.554437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1027/2042] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:56 2022] PUT /identity/v3/projects/c0ef6928958e411b9af4fd6cccafed84/users/3bcb8308b527441283b209c605cb5ca9/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:56.577547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f4843-aeb7-451a-887b-d33713da3a3f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:56.577934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f4843-aeb7-451a-887b-d33713da3a3f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:56.578279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f4843-aeb7-451a-887b-d33713da3a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:56.608416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3c9f4843-aeb7-451a-887b-d33713da3a3f None None] MFA Rules not processed for user `3bcb8308b527441283b209c605cb5ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:56.609728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3c9f4843-aeb7-451a-887b-d33713da3a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:56.685728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1016/2043] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:56.690453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c1177-e3dd-4e9d-874f-d876bf1dde1b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:56.765183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c1177-e3dd-4e9d-874f-d876bf1dde1b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:56.770225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a7b5-2828-48a9-a49a-3af0e852a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:56.770859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a7b5-2828-48a9-a49a-3af0e852a35b admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:56.771168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a7b5-2828-48a9-a49a-3af0e852a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:56.771438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a7b5-2828-48a9-a49a-3af0e852a35b admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:56.772073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85a7b5-2828-48a9-a49a-3af0e852a35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:56.774203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85a7b5-2828-48a9-a49a-3af0e852a35b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:56.890028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1028/2044] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:56 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:56.894255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f4843-aeb7-451a-887b-d33713da3a3f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:56.910017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f4843-aeb7-451a-887b-d33713da3a3f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:56.939360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:56.973873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe511-96db-409b-a7a7-77adbdaaa890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:56.975623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe511-96db-409b-a7a7-77adbdaaa890 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:56.975904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe511-96db-409b-a7a7-77adbdaaa890 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:56.976183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe511-96db-409b-a7a7-77adbdaaa890 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:56.984531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfe511-96db-409b-a7a7-77adbdaaa890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:56.984531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfe511-96db-409b-a7a7-77adbdaaa890 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:57.164031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1017/2045] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:56 2022] POST /identity/v3/users => generated 294 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:57.171952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a7b5-2828-48a9-a49a-3af0e852a35b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:57.180356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a7b5-2828-48a9-a49a-3af0e852a35b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:57.185515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d106984-f841-4f3c-948a-c9df2d6b8e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:57.186134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d106984-f841-4f3c-948a-c9df2d6b8e33 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:57.186405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d106984-f841-4f3c-948a-c9df2d6b8e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:57.186672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d106984-f841-4f3c-948a-c9df2d6b8e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:57.187349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d106984-f841-4f3c-948a-c9df2d6b8e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:57.188951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d106984-f841-4f3c-948a-c9df2d6b8e33 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:57.201388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1029/2046] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:57.231386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe511-96db-409b-a7a7-77adbdaaa890 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:57.241937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe511-96db-409b-a7a7-77adbdaaa890 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:57.247518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac94e9-fbc5-415d-b6ce-67c997343f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:57.254643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac94e9-fbc5-415d-b6ce-67c997343f3d admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:57.254643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac94e9-fbc5-415d-b6ce-67c997343f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:57.254643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac94e9-fbc5-415d-b6ce-67c997343f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:57.254643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac94e9-fbc5-415d-b6ce-67c997343f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:57.254643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac94e9-fbc5-415d-b6ce-67c997343f3d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:57.273378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1018/2047] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:57.280861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d106984-f841-4f3c-948a-c9df2d6b8e33 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:57.293715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d106984-f841-4f3c-948a-c9df2d6b8e33 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:57.299234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a3d2e-fc1d-41d4-9027-a5b56801854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:57.299876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a3d2e-fc1d-41d4-9027-a5b56801854e admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:57.300150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a3d2e-fc1d-41d4-9027-a5b56801854e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:57.300447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a3d2e-fc1d-41d4-9027-a5b56801854e admin admin] PATH_INFO: `/v3/projects/d50077f87c4a4e52a7591c9138fd9134/users/779239a5b853494ba7e97ae97837b1bf/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:57.327345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a3d2e-fc1d-41d4-9027-a5b56801854e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50077f87c4a4e52a7591c9138fd9134, user_id=779239a5b853494ba7e97ae97837b1bf, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:57.327345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a3d2e-fc1d-41d4-9027-a5b56801854e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:57.495095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1030/2048] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:57 2022] PUT /identity/v3/projects/d50077f87c4a4e52a7591c9138fd9134/users/779239a5b853494ba7e97ae97837b1bf/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:57.509560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf2cce-4a42-42ba-8534-1976edda7b4b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:57.509747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf2cce-4a42-42ba-8534-1976edda7b4b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:57.510030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf2cce-4a42-42ba-8534-1976edda7b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:57.579909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8aaf2cce-4a42-42ba-8534-1976edda7b4b None None] MFA Rules not processed for user `779239a5b853494ba7e97ae97837b1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:57.583826 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8aaf2cce-4a42-42ba-8534-1976edda7b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:57.706135 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1019/2049] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:57.710763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a3d2e-fc1d-41d4-9027-a5b56801854e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:57.775540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:57.792252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a3d2e-fc1d-41d4-9027-a5b56801854e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:57.798024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e954076-c856-454e-96a2-54365d37bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:57.798639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e954076-c856-454e-96a2-54365d37bce3 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:57.799377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e954076-c856-454e-96a2-54365d37bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:57.799377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e954076-c856-454e-96a2-54365d37bce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:57.799926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e954076-c856-454e-96a2-54365d37bce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:57.801932 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e954076-c856-454e-96a2-54365d37bce3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:57.977724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1031/2050] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:58:57 2022] POST /identity/v3/projects => generated 385 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:57.996336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf2cce-4a42-42ba-8534-1976edda7b4b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:58.013696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf2cce-4a42-42ba-8534-1976edda7b4b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:58.141195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc5878-2705-4fe1-b9f9-8e861fd3b6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:58.141195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc5878-2705-4fe1-b9f9-8e861fd3b6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:58.141195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc5878-2705-4fe1-b9f9-8e861fd3b6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:58.141195 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc5878-2705-4fe1-b9f9-8e861fd3b6c3 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:58.141522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc5878-2705-4fe1-b9f9-8e861fd3b6c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:58.141522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc5878-2705-4fe1-b9f9-8e861fd3b6c3 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:58.307723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1020/2051] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:58:57 2022] POST /identity/v3/users => generated 299 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:58:58.310613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e954076-c856-454e-96a2-54365d37bce3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:58.322517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e954076-c856-454e-96a2-54365d37bce3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:58.328459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fdfd1-86ee-4838-a3a7-e55a91a1c8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:58.329218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fdfd1-86ee-4838-a3a7-e55a91a1c8ee admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:58.332064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fdfd1-86ee-4838-a3a7-e55a91a1c8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:58.332346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fdfd1-86ee-4838-a3a7-e55a91a1c8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:58.333031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fdfd1-86ee-4838-a3a7-e55a91a1c8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:58.334658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fdfd1-86ee-4838-a3a7-e55a91a1c8ee admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:58.374126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1032/2052] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:58:58.380287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc5878-2705-4fe1-b9f9-8e861fd3b6c3 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:58.391355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc5878-2705-4fe1-b9f9-8e861fd3b6c3 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:58.396112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc92cb-4895-406e-b120-e629671af174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:58.396816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc92cb-4895-406e-b120-e629671af174 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:58.397096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc92cb-4895-406e-b120-e629671af174 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:58.397373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc92cb-4895-406e-b120-e629671af174 admin admin] PATH_INFO: `/v3/projects/015d1d6fb5294c71b8c8f821c4624dc4/users/c2f8324e6fa648fbab46996cae8f16b0/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:58.417823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc92cb-4895-406e-b120-e629671af174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=015d1d6fb5294c71b8c8f821c4624dc4, user_id=c2f8324e6fa648fbab46996cae8f16b0, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:58.420592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc92cb-4895-406e-b120-e629671af174 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:58.525088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1021/2053] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:58 2022] PUT /identity/v3/projects/015d1d6fb5294c71b8c8f821c4624dc4/users/c2f8324e6fa648fbab46996cae8f16b0/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:58.529504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fdfd1-86ee-4838-a3a7-e55a91a1c8ee admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:58.540677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fdfd1-86ee-4838-a3a7-e55a91a1c8ee admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:58.600181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f132f-31c7-4a6b-bfd6-bbb4bec99fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:58.600797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f132f-31c7-4a6b-bfd6-bbb4bec99fdb admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:58.601072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f132f-31c7-4a6b-bfd6-bbb4bec99fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:58.601342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f132f-31c7-4a6b-bfd6-bbb4bec99fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:58.601988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f132f-31c7-4a6b-bfd6-bbb4bec99fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:58.603624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f132f-31c7-4a6b-bfd6-bbb4bec99fdb admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:58.614078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1033/2054] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:58:58.618385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc92cb-4895-406e-b120-e629671af174 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:58.624481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:58:58.630346 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc92cb-4895-406e-b120-e629671af174 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:58.635767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:58.636833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:58.637112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:58.637387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] PATH_INFO: `/v3/projects/015d1d6fb5294c71b8c8f821c4624dc4/users/c2f8324e6fa648fbab46996cae8f16b0/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:58.639521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=015d1d6fb5294c71b8c8f821c4624dc4, user_id=c2f8324e6fa648fbab46996cae8f16b0, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:58.642167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:58.653760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1022/2055] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:58:58 2022] PUT /identity/v3/projects/015d1d6fb5294c71b8c8f821c4624dc4/users/c2f8324e6fa648fbab46996cae8f16b0/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:58:58.664229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33721b-0737-4e63-95ed-d2792cd6dc97 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:58.664513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33721b-0737-4e63-95ed-d2792cd6dc97 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:58.664784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33721b-0737-4e63-95ed-d2792cd6dc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:58.693812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1a33721b-0737-4e63-95ed-d2792cd6dc97 None None] MFA Rules not processed for user `c2f8324e6fa648fbab46996cae8f16b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:58:58.695133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1a33721b-0737-4e63-95ed-d2792cd6dc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:58.796982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1034/2056] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:58:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:58:58.808345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:58:58.810134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:58:58.887283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2bb58-6748-4579-b406-1dc6858fe43d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:58:58.891513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec949dd-fd93-4d5c-bfa1-8be3db6371c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:58:58.892137 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec949dd-fd93-4d5c-bfa1-8be3db6371c5 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:58:58.892408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec949dd-fd93-4d5c-bfa1-8be3db6371c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:58:58.893104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec949dd-fd93-4d5c-bfa1-8be3db6371c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:58:58.904224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec949dd-fd93-4d5c-bfa1-8be3db6371c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:58:58.905712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec949dd-fd93-4d5c-bfa1-8be3db6371c5 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:58:58.918818 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1023/2057] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:00.607217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33721b-0737-4e63-95ed-d2792cd6dc97 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:00.620392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33721b-0737-4e63-95ed-d2792cd6dc97 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:00.624420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991d4af-63f0-4b3d-8cee-7fd422ec7044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:00.625216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991d4af-63f0-4b3d-8cee-7fd422ec7044 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:00.625647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991d4af-63f0-4b3d-8cee-7fd422ec7044 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:00.626084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991d4af-63f0-4b3d-8cee-7fd422ec7044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:00.683236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:00.701385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991d4af-63f0-4b3d-8cee-7fd422ec7044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:00.702894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991d4af-63f0-4b3d-8cee-7fd422ec7044 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:00.711962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1035/2058] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:01.595730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c931e-f437-4c92-a061-c5371e800eb3 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:01.596158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c931e-f437-4c92-a061-c5371e800eb3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:01.597361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c931e-f437-4c92-a061-c5371e800eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:01.628952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-264c931e-f437-4c92-a061-c5371e800eb3 None None] MFA Rules not processed for user `b50041d36b0b430badd6442a2f09f063`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:01.631173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-264c931e-f437-4c92-a061-c5371e800eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:01.712099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1024/2059] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:11.705154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:11.884449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=10 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:16.498578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34973425-edc5-42c0-bc5d-4e44625e7624 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:16.499224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34973425-edc5-42c0-bc5d-4e44625e7624 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:16.499368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34973425-edc5-42c0-bc5d-4e44625e7624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:16.521907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-34973425-edc5-42c0-bc5d-4e44625e7624 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:16.523220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-34973425-edc5-42c0-bc5d-4e44625e7624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:16.596166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1036/2060] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 15:59:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:16.654348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c931e-f437-4c92-a061-c5371e800eb3 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:16.656818 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-264c931e-f437-4c92-a061-c5371e800eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:16.758491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c931e-f437-4c92-a061-c5371e800eb3 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:16.765675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e096765-2096-4649-b89f-269402a94be3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:16.766312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e096765-2096-4649-b89f-269402a94be3 service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:16.767188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e096765-2096-4649-b89f-269402a94be3 service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:16.767300 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e096765-2096-4649-b89f-269402a94be3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:16.781110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e096765-2096-4649-b89f-269402a94be3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:16.782749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e096765-2096-4649-b89f-269402a94be3 service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:16.799131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1025/2061] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 15:59:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:59:20.121669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34973425-edc5-42c0-bc5d-4e44625e7624 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:20.123921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-34973425-edc5-42c0-bc5d-4e44625e7624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:20.140895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34973425-edc5-42c0-bc5d-4e44625e7624 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:20.147080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:20.148818 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:20.149524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:20.149848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:20.248448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:20.251198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:20.280016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1037/2062] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:21.101310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda5797-f20d-49d8-bff0-e272c921d87b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:21.101916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda5797-f20d-49d8-bff0-e272c921d87b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:21.101916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda5797-f20d-49d8-bff0-e272c921d87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:21.143362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-bdda5797-f20d-49d8-bff0-e272c921d87b None None] MFA Rules not processed for user `4940f8a9c15a483eab251df392789534`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:21.148127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bdda5797-f20d-49d8-bff0-e272c921d87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:21.230550 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1026/2063] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:21.531285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:21.533753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:21.711383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff544ec7-e2fd-4af5-a232-e1c2d21f3039 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:21.720117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:21.721691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:21.722572 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:21.723656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] PATH_INFO: `/v3/users/50f3ad272b0e4d7db63df288c3977e47` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:21.725205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f3ad272b0e4d7db63df288c3977e47)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:21.727724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:22.065054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:22.067862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f3ad272b0e4d7db63df288c3977e47'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.071621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f3ad272b0e4d7db63df288c3977e47'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.145891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f3ad272b0e4d7db63df288c3977e47'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.318435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f3ad272b0e4d7db63df288c3977e47'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.347860 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f3ad272b0e4d7db63df288c3977e47'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:22.374837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1038/2064] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:21 2022] DELETE /identity/v3/users/50f3ad272b0e4d7db63df288c3977e47 => generated 0 bytes in 842 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:22.377625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a39cf-a21f-431d-b7e3-0526f5fd3f32 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:22.378251 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a39cf-a21f-431d-b7e3-0526f5fd3f32 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:22.378251 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a39cf-a21f-431d-b7e3-0526f5fd3f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:22.444593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e22a39cf-a21f-431d-b7e3-0526f5fd3f32 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:22.455596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e22a39cf-a21f-431d-b7e3-0526f5fd3f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:22.661557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1027/2065] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:22.670351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:22.674525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:22.840806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca685ac-daff-442a-8ee1-13d186218896 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:22.847534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b911c2-38b5-4036-9cbf-5a69090b2721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:22.849627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b911c2-38b5-4036-9cbf-5a69090b2721 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:22.850429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b911c2-38b5-4036-9cbf-5a69090b2721 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:22.851133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b911c2-38b5-4036-9cbf-5a69090b2721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:22.864665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b911c2-38b5-4036-9cbf-5a69090b2721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:22.867650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b911c2-38b5-4036-9cbf-5a69090b2721 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:22.891450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1039/2066] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:23.981307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a39cf-a21f-431d-b7e3-0526f5fd3f32 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:23.982598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e22a39cf-a21f-431d-b7e3-0526f5fd3f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:24.004816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a39cf-a21f-431d-b7e3-0526f5fd3f32 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:24.019299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:24.020292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:24.025666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:24.025666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] PATH_INFO: `/v3/projects/5efcc0d6257448329354e6a23d60f433` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:24.025666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5efcc0d6257448329354e6a23d60f433)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:24.025666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:24.337400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:24.405939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5efcc0d6257448329354e6a23d60f433', 'request_id': 'req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.429159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5efcc0d6257448329354e6a23d60f433', 'request_id': 'req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.569405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5efcc0d6257448329354e6a23d60f433', 'request_id': 'req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.609061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] The token cache is being invalidate because project 5efcc0d6257448329354e6a23d60f433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:24.609351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.614824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:24.623790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1028/2067] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:23 2022] DELETE /identity/v3/projects/5efcc0d6257448329354e6a23d60f433 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:24.749599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106491c-7e29-433f-8821-a398fec00f37 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:24.750620 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106491c-7e29-433f-8821-a398fec00f37 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:24.751189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106491c-7e29-433f-8821-a398fec00f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:24.800186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d106491c-7e29-433f-8821-a398fec00f37 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:24.801796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d106491c-7e29-433f-8821-a398fec00f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:25.024505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1040/2068] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:25.035573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:25.036853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:25.218181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112c32-7ffe-4ffa-8a08-ecc6ec7cb41d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:25.223638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f78eee-7295-48c0-b378-468734738c21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:25.224509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f78eee-7295-48c0-b378-468734738c21 service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:25.224931 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f78eee-7295-48c0-b378-468734738c21 service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:25.225522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f78eee-7295-48c0-b378-468734738c21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:25.236998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f78eee-7295-48c0-b378-468734738c21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:25.238882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f78eee-7295-48c0-b378-468734738c21 service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:25.250070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1029/2069] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 15:59:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 15:59:27.028708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106491c-7e29-433f-8821-a398fec00f37 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:27.030254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d106491c-7e29-433f-8821-a398fec00f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:27.217801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106491c-7e29-433f-8821-a398fec00f37 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:27.223790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:27.224806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:27.225986 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:27.227107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] PATH_INFO: `/v3/users/3bcb8308b527441283b209c605cb5ca9` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:27.231615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bcb8308b527441283b209c605cb5ca9)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:27.239866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:27.278935 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:27.619180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bcb8308b527441283b209c605cb5ca9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:27.619946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bcb8308b527441283b209c605cb5ca9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:27.658386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bcb8308b527441283b209c605cb5ca9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:27.879592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bcb8308b527441283b209c605cb5ca9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:27.899408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bcb8308b527441283b209c605cb5ca9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:27.923752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1041/2070] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:27 2022] DELETE /identity/v3/users/3bcb8308b527441283b209c605cb5ca9 => generated 0 bytes in 896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:27.929141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568d11e-9db2-4e30-8696-c5d13b803737 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:27.929681 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568d11e-9db2-4e30-8696-c5d13b803737 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:27.930715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568d11e-9db2-4e30-8696-c5d13b803737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:27.990616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c568d11e-9db2-4e30-8696-c5d13b803737 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:27.998981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c568d11e-9db2-4e30-8696-c5d13b803737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:28.236759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1030/2071] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:28.244379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:28.246340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:28.333230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6745c-c740-4133-8fed-342b25fd9f76 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:28.338353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083fe96-8bea-4c9a-8aaa-f71b7d427c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:28.339001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083fe96-8bea-4c9a-8aaa-f71b7d427c90 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:28.339276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083fe96-8bea-4c9a-8aaa-f71b7d427c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:28.339540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083fe96-8bea-4c9a-8aaa-f71b7d427c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:28.350553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083fe96-8bea-4c9a-8aaa-f71b7d427c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:28.352074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083fe96-8bea-4c9a-8aaa-f71b7d427c90 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:28.370082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1042/2072] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:29.693582 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568d11e-9db2-4e30-8696-c5d13b803737 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:29.695814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c568d11e-9db2-4e30-8696-c5d13b803737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:29.739187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568d11e-9db2-4e30-8696-c5d13b803737 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:29.757833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:29.758823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:29.763395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:29.763909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] PATH_INFO: `/v3/projects/c0ef6928958e411b9af4fd6cccafed84` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:29.765738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ef6928958e411b9af4fd6cccafed84)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:29.767928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:30.038224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:30.048001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ef6928958e411b9af4fd6cccafed84', 'request_id': 'req-9be935bb-f31c-4ba4-8f4d-07127797c2fb'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:30.059171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ef6928958e411b9af4fd6cccafed84', 'request_id': 'req-9be935bb-f31c-4ba4-8f4d-07127797c2fb'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:30.179421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ef6928958e411b9af4fd6cccafed84', 'request_id': 'req-9be935bb-f31c-4ba4-8f4d-07127797c2fb'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:30.231200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] The token cache is being invalidate because project c0ef6928958e411b9af4fd6cccafed84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:30.231200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:30.232469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:30.237075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1031/2073] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:29 2022] DELETE /identity/v3/projects/c0ef6928958e411b9af4fd6cccafed84 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:30.241465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083fe96-8bea-4c9a-8aaa-f71b7d427c90 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:30.258609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083fe96-8bea-4c9a-8aaa-f71b7d427c90 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:30.390464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:30.390464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:30.390464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:30.390464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] PATH_INFO: `/v3/users/779239a5b853494ba7e97ae97837b1bf` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:30.392737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779239a5b853494ba7e97ae97837b1bf)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:30.395680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:30.914797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:30.935170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779239a5b853494ba7e97ae97837b1bf'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:30.938066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779239a5b853494ba7e97ae97837b1bf'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:30.969820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779239a5b853494ba7e97ae97837b1bf'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:31.072964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779239a5b853494ba7e97ae97837b1bf'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:31.102497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779239a5b853494ba7e97ae97837b1bf'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:31.145899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1043/2074] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:30 2022] DELETE /identity/v3/users/779239a5b853494ba7e97ae97837b1bf => generated 0 bytes in 905 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:31.964281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:31.966571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:31.992882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be935bb-f31c-4ba4-8f4d-07127797c2fb admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:32.167920 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:32.168607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:32.168864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:32.169193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] PATH_INFO: `/v3/projects/d50077f87c4a4e52a7591c9138fd9134` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:32.173253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50077f87c4a4e52a7591c9138fd9134)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:32.175509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:32.484378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50077f87c4a4e52a7591c9138fd9134', 'request_id': 'req-82b16f9f-3a6b-4115-a373-9b5545b24e95'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:32.507692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50077f87c4a4e52a7591c9138fd9134', 'request_id': 'req-82b16f9f-3a6b-4115-a373-9b5545b24e95'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:32.611100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50077f87c4a4e52a7591c9138fd9134', 'request_id': 'req-82b16f9f-3a6b-4115-a373-9b5545b24e95'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:32.637559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] The token cache is being invalidate because project d50077f87c4a4e52a7591c9138fd9134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:32.637958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:32.638738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:32.640400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1032/2075] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:31 2022] DELETE /identity/v3/projects/d50077f87c4a4e52a7591c9138fd9134 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:59:32.644336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:32.646412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:32.670376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d80ab5-e39b-4ab2-9e42-0a265287d7e2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:32.775318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:32.776733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:32.777188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:32.777617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] PATH_INFO: `/v3/users/c2f8324e6fa648fbab46996cae8f16b0` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:32.779758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f8324e6fa648fbab46996cae8f16b0)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:32.782269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:33.067666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f8324e6fa648fbab46996cae8f16b0'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:33.068318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f8324e6fa648fbab46996cae8f16b0'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:33.115918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f8324e6fa648fbab46996cae8f16b0'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:33.255533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f8324e6fa648fbab46996cae8f16b0'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:33.270401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fb91275d-df01-486b-8e98-efdeb2288bb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f8324e6fa648fbab46996cae8f16b0'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:33.342090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1044/2076] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:32 2022] DELETE /identity/v3/users/c2f8324e6fa648fbab46996cae8f16b0 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:33.961678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:33.964684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:33.996924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b16f9f-3a6b-4115-a373-9b5545b24e95 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:34.202140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:34.204342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:34.206357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:34.206755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] PATH_INFO: `/v3/projects/015d1d6fb5294c71b8c8f821c4624dc4` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:34.209022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] RBAC: Authorizing `identity:delete_project(project_id=015d1d6fb5294c71b8c8f821c4624dc4)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:34.216779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:34.224116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:34.472167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015d1d6fb5294c71b8c8f821c4624dc4', 'request_id': 'req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:34.500971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015d1d6fb5294c71b8c8f821c4624dc4', 'request_id': 'req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:34.623382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015d1d6fb5294c71b8c8f821c4624dc4', 'request_id': 'req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:34.655636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] The token cache is being invalidate because project 015d1d6fb5294c71b8c8f821c4624dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:34.656027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:34.656812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d7d6497c-02eb-4c00-be9f-b0891cc5fcef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:34.681465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b6b7e-5419-4887-89e1-c256256d4c71 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:34.681465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b6b7e-5419-4887-89e1-c256256d4c71 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:34.681465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b6b7e-5419-4887-89e1-c256256d4c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:34.683746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1033/2077] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:33 2022] DELETE /identity/v3/projects/015d1d6fb5294c71b8c8f821c4624dc4 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:34.748670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-217b6b7e-5419-4887-89e1-c256256d4c71 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:34.751292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-217b6b7e-5419-4887-89e1-c256256d4c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:34.858173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:34.946151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1045/2078] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:34.957108 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349b23-5b37-4386-8fc5-c697b61df502 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:34.957705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349b23-5b37-4386-8fc5-c697b61df502 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:34.958286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349b23-5b37-4386-8fc5-c697b61df502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:35.007029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d1349b23-5b37-4386-8fc5-c697b61df502 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:35.015063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d1349b23-5b37-4386-8fc5-c697b61df502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:35.041158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1034/2079] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:59:35.044486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b6b7e-5419-4887-89e1-c256256d4c71 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:35.062907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b6b7e-5419-4887-89e1-c256256d4c71 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:35.070889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:35.072232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:35.072697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:35.073070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:35.073878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:59:35.074184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:35.083590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:35.110888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1046/2080] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:59:35.115341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1349b23-5b37-4386-8fc5-c697b61df502 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:35.133087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1349b23-5b37-4386-8fc5-c697b61df502 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:35.164674 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cc29b-fd16-4021-9615-e30773819c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:35.167753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cc29b-fd16-4021-9615-e30773819c27 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:35.168425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cc29b-fd16-4021-9615-e30773819c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:35.168866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cc29b-fd16-4021-9615-e30773819c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:35.169694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cc29b-fd16-4021-9615-e30773819c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:35.174610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cc29b-fd16-4021-9615-e30773819c27 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:35.342358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1035/2081] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:59:35 2022] POST /identity/v3/projects => generated 389 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:59:35.363546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:35.383732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212df9d-a413-4330-a6ee-80177c3e1a71 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:35.715807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ee6a6-a631-4820-ac13-80e6d134c669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:35.715807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ee6a6-a631-4820-ac13-80e6d134c669 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:35.715807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ee6a6-a631-4820-ac13-80e6d134c669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:35.715807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ee6a6-a631-4820-ac13-80e6d134c669 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:35.716729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ee6a6-a631-4820-ac13-80e6d134c669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:35.716729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ee6a6-a631-4820-ac13-80e6d134c669 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:36.373320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1047/2082] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:59:35 2022] POST /identity/v3/users => generated 295 bytes in 1010 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:59:36.379008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cc29b-fd16-4021-9615-e30773819c27 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:36.390621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cc29b-fd16-4021-9615-e30773819c27 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:36.396941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c6c5b-3cdb-4b03-8c23-eeb77e440c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:36.397158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c6c5b-3cdb-4b03-8c23-eeb77e440c84 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:36.397386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c6c5b-3cdb-4b03-8c23-eeb77e440c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:36.398059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c6c5b-3cdb-4b03-8c23-eeb77e440c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:36.398408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c6c5b-3cdb-4b03-8c23-eeb77e440c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:36.400750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c6c5b-3cdb-4b03-8c23-eeb77e440c84 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:36.416009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1036/2083] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:36.421127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ee6a6-a631-4820-ac13-80e6d134c669 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:36.432882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ee6a6-a631-4820-ac13-80e6d134c669 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:36.438758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f84c16-1904-44b6-bb38-9c562bbc1e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:36.439488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84c16-1904-44b6-bb38-9c562bbc1e3d admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:36.440626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84c16-1904-44b6-bb38-9c562bbc1e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:36.440626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84c16-1904-44b6-bb38-9c562bbc1e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:36.440860 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f84c16-1904-44b6-bb38-9c562bbc1e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:36.442557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f84c16-1904-44b6-bb38-9c562bbc1e3d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:36.466295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1048/2084] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:36.466791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c6c5b-3cdb-4b03-8c23-eeb77e440c84 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:36.522081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c6c5b-3cdb-4b03-8c23-eeb77e440c84 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:36.529261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cc180-9bc8-4235-b331-5059223aac47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:36.530027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc180-9bc8-4235-b331-5059223aac47 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:36.530706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc180-9bc8-4235-b331-5059223aac47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:36.531165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc180-9bc8-4235-b331-5059223aac47 admin admin] PATH_INFO: `/v3/projects/a357c1a638a4499bb4c9a55d0e0ce8e4/users/af5f2e4bf61645cdb8a5be48f63e23b1/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:36.592534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc180-9bc8-4235-b331-5059223aac47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a357c1a638a4499bb4c9a55d0e0ce8e4, user_id=af5f2e4bf61645cdb8a5be48f63e23b1, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:36.598403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc180-9bc8-4235-b331-5059223aac47 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:36.728688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1037/2085] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:59:36 2022] PUT /identity/v3/projects/a357c1a638a4499bb4c9a55d0e0ce8e4/users/af5f2e4bf61645cdb8a5be48f63e23b1/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:59:36.741132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951356b0-146e-4336-aa92-1019b3d145d5 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:36.741725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951356b0-146e-4336-aa92-1019b3d145d5 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:36.741725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951356b0-146e-4336-aa92-1019b3d145d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:36.797076 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-951356b0-146e-4336-aa92-1019b3d145d5 None None] MFA Rules not processed for user `af5f2e4bf61645cdb8a5be48f63e23b1`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:36.798263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-951356b0-146e-4336-aa92-1019b3d145d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:37.047882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1049/2086] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:37.051842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cc180-9bc8-4235-b331-5059223aac47 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:37.243879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cc180-9bc8-4235-b331-5059223aac47 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:37.254496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c85dd-0c67-4bd7-b69d-dbacfe9401b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:37.255971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c85dd-0c67-4bd7-b69d-dbacfe9401b0 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:37.256585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c85dd-0c67-4bd7-b69d-dbacfe9401b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:37.257249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c85dd-0c67-4bd7-b69d-dbacfe9401b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:37.258173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c85dd-0c67-4bd7-b69d-dbacfe9401b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:37.262540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c85dd-0c67-4bd7-b69d-dbacfe9401b0 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:37.406383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951356b0-146e-4336-aa92-1019b3d145d5 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:37.407209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1038/2087] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:59:37 2022] POST /identity/v3/projects => generated 385 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:59:37.428987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951356b0-146e-4336-aa92-1019b3d145d5 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:37.516694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:37.620615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07de7f0-e932-4671-ad56-5b6d6d73972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:37.621612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07de7f0-e932-4671-ad56-5b6d6d73972d admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:37.622213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07de7f0-e932-4671-ad56-5b6d6d73972d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:37.623474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07de7f0-e932-4671-ad56-5b6d6d73972d admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:37.624242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07de7f0-e932-4671-ad56-5b6d6d73972d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:37.627909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07de7f0-e932-4671-ad56-5b6d6d73972d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:37.769127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1050/2088] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:59:37 2022] POST /identity/v3/users => generated 299 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:59:37.771387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c85dd-0c67-4bd7-b69d-dbacfe9401b0 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:37.795936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c85dd-0c67-4bd7-b69d-dbacfe9401b0 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:37.820172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4292b-c220-451e-a311-1f755890dc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:37.822236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4292b-c220-451e-a311-1f755890dc6e admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:37.822664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4292b-c220-451e-a311-1f755890dc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:37.822889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4292b-c220-451e-a311-1f755890dc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:37.823491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4292b-c220-451e-a311-1f755890dc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:37.831112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4292b-c220-451e-a311-1f755890dc6e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:37.852467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1039/2089] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:37.858297 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07de7f0-e932-4671-ad56-5b6d6d73972d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:37.878603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07de7f0-e932-4671-ad56-5b6d6d73972d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:37.887194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cb7fc-9873-4ac5-a71d-381332ba6025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:37.888011 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb7fc-9873-4ac5-a71d-381332ba6025 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:37.888396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb7fc-9873-4ac5-a71d-381332ba6025 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:37.888769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cb7fc-9873-4ac5-a71d-381332ba6025 admin admin] PATH_INFO: `/v3/projects/3fe4d8c0c41541b5b9ddf2dce70cf351/users/42bbc886bb6440f6a62855e55d3bc0e9/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:37.910633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cb7fc-9873-4ac5-a71d-381332ba6025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe4d8c0c41541b5b9ddf2dce70cf351, user_id=42bbc886bb6440f6a62855e55d3bc0e9, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:37.914511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cb7fc-9873-4ac5-a71d-381332ba6025 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:38.062879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1051/2090] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:59:37 2022] PUT /identity/v3/projects/3fe4d8c0c41541b5b9ddf2dce70cf351/users/42bbc886bb6440f6a62855e55d3bc0e9/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 15:59:38.067150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4292b-c220-451e-a311-1f755890dc6e admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:38.075220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:38.087667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4292b-c220-451e-a311-1f755890dc6e admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:38.360246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383ec1b-78e9-49a8-80fa-9fe6cff86994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:38.361342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ec1b-78e9-49a8-80fa-9fe6cff86994 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:38.362011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ec1b-78e9-49a8-80fa-9fe6cff86994 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:38.362573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ec1b-78e9-49a8-80fa-9fe6cff86994 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:38.364368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383ec1b-78e9-49a8-80fa-9fe6cff86994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:38.370008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383ec1b-78e9-49a8-80fa-9fe6cff86994 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:38.418643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1040/2091] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:38.421767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cb7fc-9873-4ac5-a71d-381332ba6025 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:38.432917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cb7fc-9873-4ac5-a71d-381332ba6025 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:38.456715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:38.458247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:38.460475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:38.461012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] PATH_INFO: `/v3/projects/3fe4d8c0c41541b5b9ddf2dce70cf351/users/42bbc886bb6440f6a62855e55d3bc0e9/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:38.471272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe4d8c0c41541b5b9ddf2dce70cf351, user_id=42bbc886bb6440f6a62855e55d3bc0e9, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:38.474398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:38.508373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1052/2092] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:59:38 2022] PUT /identity/v3/projects/3fe4d8c0c41541b5b9ddf2dce70cf351/users/42bbc886bb6440f6a62855e55d3bc0e9/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:38.527779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ac5f6-d765-45c7-a792-0412b93dca8b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:38.528392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ac5f6-d765-45c7-a792-0412b93dca8b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:38.528813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ac5f6-d765-45c7-a792-0412b93dca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:38.644048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d15ac5f6-d765-45c7-a792-0412b93dca8b None None] MFA Rules not processed for user `42bbc886bb6440f6a62855e55d3bc0e9`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:38.651363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d15ac5f6-d765-45c7-a792-0412b93dca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:38.923149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1041/2093] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:38.929819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:38.933856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:39.181220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbb06a-9381-45c2-975d-30bf238422fd admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:39.198004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c145b-f7fc-405a-b164-93c03e4859f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:39.198004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c145b-f7fc-405a-b164-93c03e4859f6 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:39.198004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c145b-f7fc-405a-b164-93c03e4859f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:39.205358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c145b-f7fc-405a-b164-93c03e4859f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:39.258763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c145b-f7fc-405a-b164-93c03e4859f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:39.260265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c145b-f7fc-405a-b164-93c03e4859f6 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:39.344971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1053/2094] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:39.577830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ac5f6-d765-45c7-a792-0412b93dca8b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:39.727299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ac5f6-d765-45c7-a792-0412b93dca8b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:39.753339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:39.754130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:39.754504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:39.754781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] PATH_INFO: `/v3/users/af5f2e4bf61645cdb8a5be48f63e23b1` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:39.762048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5f2e4bf61645cdb8a5be48f63e23b1)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:39.764244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:39.900433 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:40.131375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5f2e4bf61645cdb8a5be48f63e23b1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:40.132460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5f2e4bf61645cdb8a5be48f63e23b1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:40.198948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5f2e4bf61645cdb8a5be48f63e23b1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:40.229042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5f2e4bf61645cdb8a5be48f63e23b1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:40.333798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5f2e4bf61645cdb8a5be48f63e23b1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:40.398609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1042/2095] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:39 2022] DELETE /identity/v3/users/af5f2e4bf61645cdb8a5be48f63e23b1 => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:40.401958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028f944-5ebf-451e-a0d3-628ad4647c3e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:40.403788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028f944-5ebf-451e-a0d3-628ad4647c3e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:40.404536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028f944-5ebf-451e-a0d3-628ad4647c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:40.436138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d028f944-5ebf-451e-a0d3-628ad4647c3e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:40.437295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d028f944-5ebf-451e-a0d3-628ad4647c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:40.554061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1054/2096] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:40.564960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:40.567360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:40.652044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4680b2-0873-4669-9886-23e00bc24469 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:40.658522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df387c2-7675-4159-9e94-0d0547ee5647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:40.659822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df387c2-7675-4159-9e94-0d0547ee5647 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:40.660151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df387c2-7675-4159-9e94-0d0547ee5647 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:40.660528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df387c2-7675-4159-9e94-0d0547ee5647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:40.682406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df387c2-7675-4159-9e94-0d0547ee5647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:40.684010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df387c2-7675-4159-9e94-0d0547ee5647 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:40.703578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1043/2097] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:42.213569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028f944-5ebf-451e-a0d3-628ad4647c3e None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:42.215686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d028f944-5ebf-451e-a0d3-628ad4647c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:42.230828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028f944-5ebf-451e-a0d3-628ad4647c3e None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:42.236867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:42.237554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:42.237828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:42.238034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] PATH_INFO: `/v3/projects/a357c1a638a4499bb4c9a55d0e0ce8e4` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:42.239194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a357c1a638a4499bb4c9a55d0e0ce8e4)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:42.241105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:42.262861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:42.362335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a357c1a638a4499bb4c9a55d0e0ce8e4', 'request_id': 'req-ad31af21-b0dd-452a-996e-3de23c9189aa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:42.375047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a357c1a638a4499bb4c9a55d0e0ce8e4', 'request_id': 'req-ad31af21-b0dd-452a-996e-3de23c9189aa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:42.495222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a357c1a638a4499bb4c9a55d0e0ce8e4', 'request_id': 'req-ad31af21-b0dd-452a-996e-3de23c9189aa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:42.519196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] The token cache is being invalidate because project a357c1a638a4499bb4c9a55d0e0ce8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:42.519871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:42.521387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:42.528274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df387c2-7675-4159-9e94-0d0547ee5647 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:42.529722 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1055/2098] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:42 2022] DELETE /identity/v3/projects/a357c1a638a4499bb4c9a55d0e0ce8e4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:42.548065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df387c2-7675-4159-9e94-0d0547ee5647 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:42.766170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:42.767309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:42.767580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:42.767997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] PATH_INFO: `/v3/users/42bbc886bb6440f6a62855e55d3bc0e9` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:42.771988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42bbc886bb6440f6a62855e55d3bc0e9)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:42.775852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:43.216292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42bbc886bb6440f6a62855e55d3bc0e9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:43.217059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42bbc886bb6440f6a62855e55d3bc0e9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:43.219579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:43.261766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42bbc886bb6440f6a62855e55d3bc0e9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:43.263342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42bbc886bb6440f6a62855e55d3bc0e9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:43.412635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2c500b2d-1614-47e7-835a-e10ba93e0c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42bbc886bb6440f6a62855e55d3bc0e9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:43.460591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1044/2099] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:42 2022] DELETE /identity/v3/users/42bbc886bb6440f6a62855e55d3bc0e9 => generated 0 bytes in 930 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:44.545089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:44.546630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:44.564492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31af21-b0dd-452a-996e-3de23c9189aa admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:44.673827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:44.674535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:44.674815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:44.675485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] PATH_INFO: `/v3/projects/3fe4d8c0c41541b5b9ddf2dce70cf351` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:44.677080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe4d8c0c41541b5b9ddf2dce70cf351)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:44.680948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:44.851992 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe4d8c0c41541b5b9ddf2dce70cf351', 'request_id': 'req-76d48a8c-fd59-47f0-92e7-73b321f8e113'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:44.874824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe4d8c0c41541b5b9ddf2dce70cf351', 'request_id': 'req-76d48a8c-fd59-47f0-92e7-73b321f8e113'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:45.014108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe4d8c0c41541b5b9ddf2dce70cf351', 'request_id': 'req-76d48a8c-fd59-47f0-92e7-73b321f8e113'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:45.028912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] The token cache is being invalidate because project 3fe4d8c0c41541b5b9ddf2dce70cf351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:45.029210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:45.030849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:45.032581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1056/2100] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:44 2022] DELETE /identity/v3/projects/3fe4d8c0c41541b5b9ddf2dce70cf351 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:45.038631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fde0b-b9a6-4cfa-942f-645467c14c54 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.039314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fde0b-b9a6-4cfa-942f-645467c14c54 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.039412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fde0b-b9a6-4cfa-942f-645467c14c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.061534 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-130fde0b-b9a6-4cfa-942f-645467c14c54 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:45.063676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-130fde0b-b9a6-4cfa-942f-645467c14c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:45.098974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:45.102187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:45.163562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1045/2101] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:45.166053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564632f6-2533-46a7-9283-6829f5697be3 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.166345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564632f6-2533-46a7-9283-6829f5697be3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.166616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564632f6-2533-46a7-9283-6829f5697be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.188731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-564632f6-2533-46a7-9283-6829f5697be3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:45.189952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-564632f6-2533-46a7-9283-6829f5697be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:45.191741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:45.207567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1046/2102] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:59:45.209517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564632f6-2533-46a7-9283-6829f5697be3 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:45.224995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564632f6-2533-46a7-9283-6829f5697be3 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:45.232554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:45.234078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.234543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.235064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.235981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:59:45.236447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:45.241686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:45.245029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d48a8c-fd59-47f0-92e7-73b321f8e113 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:45.251586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:45.252225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.252504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.254007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.254879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:45.270383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:45.272342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:45.279244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1047/2103] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:59:45.281812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:45.288385 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1057/2104] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:45.299059 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe72e3-e2b7-4410-841b-caed4ddf9977 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:45.306207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcebeaa-ed46-4297-aff9-5af8b3339b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:45.307476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcebeaa-ed46-4297-aff9-5af8b3339b42 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.307722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcebeaa-ed46-4297-aff9-5af8b3339b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.313365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcebeaa-ed46-4297-aff9-5af8b3339b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.314635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcebeaa-ed46-4297-aff9-5af8b3339b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:45.316405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcebeaa-ed46-4297-aff9-5af8b3339b42 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:45.425057 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1048/2105] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:59:45 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 15:59:45.425776 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:45.443245 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d4a99-0b58-4575-85b1-3f1185da65f7 service placement] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:45.527972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215ae21-047a-492d-9309-6e52d56a5d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:45.528514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215ae21-047a-492d-9309-6e52d56a5d8c admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.528978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215ae21-047a-492d-9309-6e52d56a5d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.529258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215ae21-047a-492d-9309-6e52d56a5d8c admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.529979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215ae21-047a-492d-9309-6e52d56a5d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:45.531886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215ae21-047a-492d-9309-6e52d56a5d8c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:45.541562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c358ec-8968-4a41-bb21-3083196cfb93 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.542678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c358ec-8968-4a41-bb21-3083196cfb93 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.542988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c358ec-8968-4a41-bb21-3083196cfb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.566391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-41c358ec-8968-4a41-bb21-3083196cfb93 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:45.567736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-41c358ec-8968-4a41-bb21-3083196cfb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:45.632679 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1049/2106] 10.222.0.94 () {58 vars in 967 bytes} [Fri Aug 26 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:59:45.645760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c358ec-8968-4a41-bb21-3083196cfb93 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:45.647324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-41c358ec-8968-4a41-bb21-3083196cfb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:45.662184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1058/2107] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:59:45 2022] POST /identity/v3/users => generated 288 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:59:45.667514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215ae21-047a-492d-9309-6e52d56a5d8c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:45.678528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215ae21-047a-492d-9309-6e52d56a5d8c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:45.684999 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3c666-0cf6-4cc1-8adc-b026d17f8902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:45.685851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:45.686259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3c666-0cf6-4cc1-8adc-b026d17f8902 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.686554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3c666-0cf6-4cc1-8adc-b026d17f8902 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.686996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3c666-0cf6-4cc1-8adc-b026d17f8902 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.687847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3c666-0cf6-4cc1-8adc-b026d17f8902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:45.690698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3c666-0cf6-4cc1-8adc-b026d17f8902 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:45.706467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1059/2108] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:45.710718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3c666-0cf6-4cc1-8adc-b026d17f8902 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:45.723325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3c666-0cf6-4cc1-8adc-b026d17f8902 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:45.725496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c358ec-8968-4a41-bb21-3083196cfb93 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:45.730098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0a5ad-284f-4839-a49b-5f5c38faa1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:45.731015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0a5ad-284f-4839-a49b-5f5c38faa1ec admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.731446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0a5ad-284f-4839-a49b-5f5c38faa1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.731580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf908ef5-b8d7-43aa-9068-91b32c2a047c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:45.732024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0a5ad-284f-4839-a49b-5f5c38faa1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.732155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908ef5-b8d7-43aa-9068-91b32c2a047c service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.732418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908ef5-b8d7-43aa-9068-91b32c2a047c service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.732684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908ef5-b8d7-43aa-9068-91b32c2a047c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.733347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0a5ad-284f-4839-a49b-5f5c38faa1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:45.735315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0a5ad-284f-4839-a49b-5f5c38faa1ec admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:45.741439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf908ef5-b8d7-43aa-9068-91b32c2a047c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:45.743767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf908ef5-b8d7-43aa-9068-91b32c2a047c service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:45.750735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1060/2109] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 15:59:45.756446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0a5ad-284f-4839-a49b-5f5c38faa1ec admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:45.759178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1050/2110] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 15:59:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 15:59:45.768797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0a5ad-284f-4839-a49b-5f5c38faa1ec admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:45.774040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:45.785619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.785619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.785619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] PATH_INFO: `/v3/projects/aa98d1bd1c354b38b99eb85b741e44ab/users/796e6d2c1d3a4ef59c464fb2e61affe5/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.793620 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa98d1bd1c354b38b99eb85b741e44ab, user_id=796e6d2c1d3a4ef59c464fb2e61affe5, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:45.796785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:45.943403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1061/2111] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:59:45 2022] PUT /identity/v3/projects/aa98d1bd1c354b38b99eb85b741e44ab/users/796e6d2c1d3a4ef59c464fb2e61affe5/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:45.954540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adedb19-f123-4a65-90cd-a793761bf587 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:45.954822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adedb19-f123-4a65-90cd-a793761bf587 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:45.955175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adedb19-f123-4a65-90cd-a793761bf587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:45.985797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0adedb19-f123-4a65-90cd-a793761bf587 None None] MFA Rules not processed for user `796e6d2c1d3a4ef59c464fb2e61affe5`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:45.987245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0adedb19-f123-4a65-90cd-a793761bf587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:46.083453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1051/2112] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:46.098067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:46.099801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:46.162224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ef436-ed78-42e9-8428-624189af8594 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:46.166698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:46.167342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:46.167634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:46.167968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:46.176430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:46.177854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:46.186998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1062/2113] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 15:59:49.093143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7598d5f-3b8d-4af7-b37d-cfaa784bd9f0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:49.093753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7598d5f-3b8d-4af7-b37d-cfaa784bd9f0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:49.093753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7598d5f-3b8d-4af7-b37d-cfaa784bd9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:49.122947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e7598d5f-3b8d-4af7-b37d-cfaa784bd9f0 None None] MFA Rules not processed for user `796e6d2c1d3a4ef59c464fb2e61affe5`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:49.122947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e7598d5f-3b8d-4af7-b37d-cfaa784bd9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:49.123590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1052/2114] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:50.830838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:50.833564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:50.857061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:50.922296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37531c-03e5-4353-9b6c-9e3ebc890def service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:50.927982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:50.928675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:50.929452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:50.929920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] PATH_INFO: `/v3/users/abe845d55b4b4ac898556dbdc264c086` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:50.932339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe845d55b4b4ac898556dbdc264c086)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:50.934398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:51.161735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe845d55b4b4ac898556dbdc264c086'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:51.162658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe845d55b4b4ac898556dbdc264c086'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:51.181491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe845d55b4b4ac898556dbdc264c086'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:51.283775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe845d55b4b4ac898556dbdc264c086'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:51.295948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-5a4718e9-3e64-49e0-aada-40f6045c0e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe845d55b4b4ac898556dbdc264c086'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:51.304184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1063/2115] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 15:59:50 2022] DELETE /identity/v3/users/abe845d55b4b4ac898556dbdc264c086 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:51.851003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7598d5f-3b8d-4af7-b37d-cfaa784bd9f0 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:51.851003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7598d5f-3b8d-4af7-b37d-cfaa784bd9f0 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:51.851003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:51.851003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:51.855396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:51.855396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] PATH_INFO: `/v3/projects/2ff1702cbe6d41b196df47d78f148368` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:51.855396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff1702cbe6d41b196df47d78f148368)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:51.855396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:51.977365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:51.981316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff1702cbe6d41b196df47d78f148368', 'request_id': 'req-0792acd5-c19c-4476-88e5-f771ce3286d1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:51.987822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff1702cbe6d41b196df47d78f148368', 'request_id': 'req-0792acd5-c19c-4476-88e5-f771ce3286d1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.089689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff1702cbe6d41b196df47d78f148368', 'request_id': 'req-0792acd5-c19c-4476-88e5-f771ce3286d1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.105421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] The token cache is being invalidate because project 2ff1702cbe6d41b196df47d78f148368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 15:59:52.106055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.107053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0792acd5-c19c-4476-88e5-f771ce3286d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 15:59:52.109365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1053/2116] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 15:59:51 2022] DELETE /identity/v3/projects/2ff1702cbe6d41b196df47d78f148368 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:52.118331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdae44a-e44f-4071-a4b0-1b0a2be55b6c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.118331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdae44a-e44f-4071-a4b0-1b0a2be55b6c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.118331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdae44a-e44f-4071-a4b0-1b0a2be55b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.137835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5fdae44a-e44f-4071-a4b0-1b0a2be55b6c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:52.143293 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5fdae44a-e44f-4071-a4b0-1b0a2be55b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:52.229804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1064/2117] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:52.238214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb352f-b24d-43fc-82f2-a62655cd15f7 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.238571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb352f-b24d-43fc-82f2-a62655cd15f7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.238785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb352f-b24d-43fc-82f2-a62655cd15f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.261427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f3bb352f-b24d-43fc-82f2-a62655cd15f7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:52.262869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f3bb352f-b24d-43fc-82f2-a62655cd15f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:52.273262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1054/2118] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 15:59:52.278805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdae44a-e44f-4071-a4b0-1b0a2be55b6c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:52.290033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdae44a-e44f-4071-a4b0-1b0a2be55b6c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:52.295874 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:52.296588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.296897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.297173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.297948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 15:59:52.298319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:52.300105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:52.315098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1065/2119] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 15:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 15:59:52.318455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb352f-b24d-43fc-82f2-a62655cd15f7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:52.329302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb352f-b24d-43fc-82f2-a62655cd15f7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:52.334119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495213e7-07f4-465f-be4c-b85e1ec10cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:52.334727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495213e7-07f4-465f-be4c-b85e1ec10cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.335028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495213e7-07f4-465f-be4c-b85e1ec10cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.335303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495213e7-07f4-465f-be4c-b85e1ec10cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.335931 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495213e7-07f4-465f-be4c-b85e1ec10cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:52.337901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495213e7-07f4-465f-be4c-b85e1ec10cc7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:52.477648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1055/2120] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 15:59:52 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:59:52.479166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:52.493657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac43f-4ad6-49be-a284-1a9ef7003966 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:52.511654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 15:59:52.563796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04486428-d204-48aa-924e-c33db7639db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:52.564718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04486428-d204-48aa-924e-c33db7639db4 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.564994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04486428-d204-48aa-924e-c33db7639db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.565279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04486428-d204-48aa-924e-c33db7639db4 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.565901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04486428-d204-48aa-924e-c33db7639db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:52.567893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04486428-d204-48aa-924e-c33db7639db4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:52.702340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1066/2121] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 15:59:52 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 15:59:52.706493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495213e7-07f4-465f-be4c-b85e1ec10cc7 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:52.719514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495213e7-07f4-465f-be4c-b85e1ec10cc7 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:52.724552 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966443a-bcb7-49da-a8ae-fc03e162f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:52.725155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966443a-bcb7-49da-a8ae-fc03e162f91d admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.725431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966443a-bcb7-49da-a8ae-fc03e162f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.725701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966443a-bcb7-49da-a8ae-fc03e162f91d admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.726344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966443a-bcb7-49da-a8ae-fc03e162f91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:52.728599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966443a-bcb7-49da-a8ae-fc03e162f91d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:52.740835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1056/2122] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:52.745485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04486428-d204-48aa-924e-c33db7639db4 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:52.756247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04486428-d204-48aa-924e-c33db7639db4 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:52.761521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a18416-975b-495f-8873-efce0ffa9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:52.762163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a18416-975b-495f-8873-efce0ffa9326 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.762472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a18416-975b-495f-8873-efce0ffa9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.762650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a18416-975b-495f-8873-efce0ffa9326 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.763325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a18416-975b-495f-8873-efce0ffa9326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:52.764924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a18416-975b-495f-8873-efce0ffa9326 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:52.777063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1067/2123] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 15:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 15:59:52.781429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966443a-bcb7-49da-a8ae-fc03e162f91d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:52.792622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966443a-bcb7-49da-a8ae-fc03e162f91d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:52.797575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:52.798235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.798513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.798812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] PATH_INFO: `/v3/projects/1429c284e8e748058de2dae09d68680e/users/21522dde4b5b471eb554d9b955716fec/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.816386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1429c284e8e748058de2dae09d68680e, user_id=21522dde4b5b471eb554d9b955716fec, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:52.819067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:52.924250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1057/2124] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 15:59:52 2022] PUT /identity/v3/projects/1429c284e8e748058de2dae09d68680e/users/21522dde4b5b471eb554d9b955716fec/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 15:59:52.935964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7108e-23ba-415f-89ca-d535b95e1e52 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:52.936453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7108e-23ba-415f-89ca-d535b95e1e52 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:52.936931 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7108e-23ba-415f-89ca-d535b95e1e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:52.967825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-52c7108e-23ba-415f-89ca-d535b95e1e52 None None] MFA Rules not processed for user `21522dde4b5b471eb554d9b955716fec`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 15:59:52.968987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-52c7108e-23ba-415f-89ca-d535b95e1e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:53.071593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1068/2125] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 15:59:53.082911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 15:59:53.084459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 15:59:53.156853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454494ce-881f-4990-86f0-2eb624efc676 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 15:59:53.161608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468ecfa-3154-4843-988d-517dda03636c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23ddf8f600454da7ae9b715cd4cbd186', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 15:59:53.162311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ecfa-3154-4843-988d-517dda03636c service glance] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 15:59:53.162690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ecfa-3154-4843-988d-517dda03636c service glance] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 15:59:53.163102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468ecfa-3154-4843-988d-517dda03636c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 15:59:53.173997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468ecfa-3154-4843-988d-517dda03636c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 15:59:53.175848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468ecfa-3154-4843-988d-517dda03636c service glance] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 15:59:53.187796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1058/2126] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 15:59:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:00.094752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9af648-81a6-4d62-b55e-e4162cbb00c0 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:00.095774 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9af648-81a6-4d62-b55e-e4162cbb00c0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:00.096532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9af648-81a6-4d62-b55e-e4162cbb00c0 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:00.098692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1069/2127] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 16:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 16:00:00.109865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a93c-4e2a-4794-b4b3-2c8aa72757c3 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:00.110425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a93c-4e2a-4794-b4b3-2c8aa72757c3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:00.117685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a93c-4e2a-4794-b4b3-2c8aa72757c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:00.155040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5eb8a93c-4e2a-4794-b4b3-2c8aa72757c3 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:00.157161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5eb8a93c-4e2a-4794-b4b3-2c8aa72757c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:00.275169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1059/2128] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:00.289530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9af648-81a6-4d62-b55e-e4162cbb00c0 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:00.291255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9f9af648-81a6-4d62-b55e-e4162cbb00c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:00.305981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9af648-81a6-4d62-b55e-e4162cbb00c0 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:00.310707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df811e-86a8-4f4d-8ae7-ac2c12fc7872 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23ddf8f600454da7ae9b715cd4cbd186', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:00.318398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df811e-86a8-4f4d-8ae7-ac2c12fc7872 service glance] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:00.318398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df811e-86a8-4f4d-8ae7-ac2c12fc7872 service glance] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:00.318398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df811e-86a8-4f4d-8ae7-ac2c12fc7872 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:00.326341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df811e-86a8-4f4d-8ae7-ac2c12fc7872 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:00.327885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df811e-86a8-4f4d-8ae7-ac2c12fc7872 service glance] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:00.352519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1070/2129] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:00.632110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c051cc-5388-4a03-b43f-e71e39284664 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:00.632473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c051cc-5388-4a03-b43f-e71e39284664 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:00.632683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c051cc-5388-4a03-b43f-e71e39284664 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:00.634438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1060/2130] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 16:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 16:00:00.641388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf947-f5ab-4614-b72f-7562c12088b2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:00.641809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf947-f5ab-4614-b72f-7562c12088b2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:00.642241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf947-f5ab-4614-b72f-7562c12088b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:00.664169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cdcaf947-f5ab-4614-b72f-7562c12088b2 None None] MFA Rules not processed for user `b7ad758153334ec28940156d09495ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:00.666664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cdcaf947-f5ab-4614-b72f-7562c12088b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:00.751420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1071/2131] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:00.768153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c051cc-5388-4a03-b43f-e71e39284664 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:00.770819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c7c051cc-5388-4a03-b43f-e71e39284664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:00.780372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:00.792274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c051cc-5388-4a03-b43f-e71e39284664 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:00.809375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102f5a2-6eca-4877-8e2a-36b73d685ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:00.809375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102f5a2-6eca-4877-8e2a-36b73d685ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:00.810500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102f5a2-6eca-4877-8e2a-36b73d685ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:00.810781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102f5a2-6eca-4877-8e2a-36b73d685ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:00.834869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102f5a2-6eca-4877-8e2a-36b73d685ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:00.834869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102f5a2-6eca-4877-8e2a-36b73d685ad3 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:00.860509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1061/2132] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:00.981286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8aeec-4887-4fe0-9e17-719fc4961b5a None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:00.981975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8aeec-4887-4fe0-9e17-719fc4961b5a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:00.982127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8aeec-4887-4fe0-9e17-719fc4961b5a None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:00.984596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1072/2133] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 16:00:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 16:00:00.991803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc68522-49df-4072-9788-275a2a0e70e6 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:00.995888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc68522-49df-4072-9788-275a2a0e70e6 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:00.995888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc68522-49df-4072-9788-275a2a0e70e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:01.029695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0fc68522-49df-4072-9788-275a2a0e70e6 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:01.037187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0fc68522-49df-4072-9788-275a2a0e70e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:01.053897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1062/2134] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:01.084434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8aeec-4887-4fe0-9e17-719fc4961b5a None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:01.086295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-caa8aeec-4887-4fe0-9e17-719fc4961b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:01.106507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8aeec-4887-4fe0-9e17-719fc4961b5a None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:01.110309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:01.111126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:01.111608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:01.111870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:01.125488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:01.127087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:01.141537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1073/2135] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:10.319608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d407-f43a-4b2a-8726-c2e432489c32 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:10.320196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d407-f43a-4b2a-8726-c2e432489c32 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:10.320196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d407-f43a-4b2a-8726-c2e432489c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:10.346438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-18e2d407-f43a-4b2a-8726-c2e432489c32 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:10.346438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-18e2d407-f43a-4b2a-8726-c2e432489c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:10.424206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1063/2136] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 16:00:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:10.426633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:10.428122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:10.442209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:10.503471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a2f4b-b52b-4b30-a4c0-7e4e9ce81477 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:10.508626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6544a9-b6bf-4a38-b59b-ea98735506d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:10.509246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6544a9-b6bf-4a38-b59b-ea98735506d4 service swift] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:10.509552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6544a9-b6bf-4a38-b59b-ea98735506d4 service swift] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:10.509866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6544a9-b6bf-4a38-b59b-ea98735506d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:10.518525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6544a9-b6bf-4a38-b59b-ea98735506d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:10.520150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6544a9-b6bf-4a38-b59b-ea98735506d4 service swift] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:10.529016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1074/2137] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:00:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:00:20.414231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=33 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:20.637166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=28 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:20.781414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bc38-f360-4c17-9162-ae170aee3426 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:20.781691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bc38-f360-4c17-9162-ae170aee3426 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:20.781957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bc38-f360-4c17-9162-ae170aee3426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:20.801936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ed91bc38-f360-4c17-9162-ae170aee3426 None None] MFA Rules not processed for user `a0ecfd3d64ba4fef820a31b1522125ff`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:20.803258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ed91bc38-f360-4c17-9162-ae170aee3426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:20.865899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1064/2138] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:30.858223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=25 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:36.285565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d140a-759d-498e-94f1-721773e0633d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:36.286584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d140a-759d-498e-94f1-721773e0633d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:36.287335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d140a-759d-498e-94f1-721773e0633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:36.316471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-aa0d140a-759d-498e-94f1-721773e0633d None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:36.316471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-aa0d140a-759d-498e-94f1-721773e0633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:36.323552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1075/2139] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 16:00:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:36.350525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91bc38-f360-4c17-9162-ae170aee3426 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:36.360027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91bc38-f360-4c17-9162-ae170aee3426 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:36.365121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e238d-9390-4f0f-98c4-d476af7755a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:36.365784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e238d-9390-4f0f-98c4-d476af7755a3 service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:36.366054 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e238d-9390-4f0f-98c4-d476af7755a3 service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:36.366314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e238d-9390-4f0f-98c4-d476af7755a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:36.376804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e238d-9390-4f0f-98c4-d476af7755a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:36.377460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e238d-9390-4f0f-98c4-d476af7755a3 service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:36.386645 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1065/2140] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:00:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:00:37.304439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f902e96-29f4-4e87-8092-f6147e731845 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.304953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f902e96-29f4-4e87-8092-f6147e731845 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.305142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f902e96-29f4-4e87-8092-f6147e731845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.326882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2f902e96-29f4-4e87-8092-f6147e731845 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:37.328079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2f902e96-29f4-4e87-8092-f6147e731845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:37.335363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1076/2141] 10.222.0.94 () {58 vars in 967 bytes} [Fri Aug 26 16:00:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:37.350308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e238d-9390-4f0f-98c4-d476af7755a3 service swift] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:37.362336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e238d-9390-4f0f-98c4-d476af7755a3 service swift] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:37.367728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f1116-84ef-4b51-8587-04cb3012d855 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:37.368271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f1116-84ef-4b51-8587-04cb3012d855 service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:37.368668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f1116-84ef-4b51-8587-04cb3012d855 service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:37.368946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f1116-84ef-4b51-8587-04cb3012d855 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:37.378410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f1116-84ef-4b51-8587-04cb3012d855 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:37.380158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f1116-84ef-4b51-8587-04cb3012d855 service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:37.389573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1066/2142] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:00:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:00:38.225107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158db17-0a9c-49ac-be4b-3d5821fb8280 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:38.225378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158db17-0a9c-49ac-be4b-3d5821fb8280 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:38.225836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158db17-0a9c-49ac-be4b-3d5821fb8280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:38.246563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3158db17-0a9c-49ac-be4b-3d5821fb8280 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:38.247873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3158db17-0a9c-49ac-be4b-3d5821fb8280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:38.258802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1077/2143] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 16:00:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:38.272271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f1116-84ef-4b51-8587-04cb3012d855 service swift] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:38.283801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f1116-84ef-4b51-8587-04cb3012d855 service swift] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:38.289573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b276a8e-680c-44fd-8edd-2114f9a4bc6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:38.290320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b276a8e-680c-44fd-8edd-2114f9a4bc6c service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:38.290591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b276a8e-680c-44fd-8edd-2114f9a4bc6c service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:38.290867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b276a8e-680c-44fd-8edd-2114f9a4bc6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:38.301584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b276a8e-680c-44fd-8edd-2114f9a4bc6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:38.303361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b276a8e-680c-44fd-8edd-2114f9a4bc6c service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:38.314554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1067/2144] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:00:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:00:39.091182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e436e18-a6d5-4440-80d8-1873924e5a62 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:39.092129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e436e18-a6d5-4440-80d8-1873924e5a62 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:39.092563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e436e18-a6d5-4440-80d8-1873924e5a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:39.114234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3e436e18-a6d5-4440-80d8-1873924e5a62 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:39.115641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3e436e18-a6d5-4440-80d8-1873924e5a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:39.133420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1078/2145] 10.222.0.94 () {58 vars in 967 bytes} [Fri Aug 26 16:00:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:39.154425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b276a8e-680c-44fd-8edd-2114f9a4bc6c service swift] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:39.165687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b276a8e-680c-44fd-8edd-2114f9a4bc6c service swift] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:39.171869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b24eb-a02f-4eaa-9573-e5f6dab8b4df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:39.172576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b24eb-a02f-4eaa-9573-e5f6dab8b4df service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:39.172941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b24eb-a02f-4eaa-9573-e5f6dab8b4df service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:39.173280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b24eb-a02f-4eaa-9573-e5f6dab8b4df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:39.181956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b24eb-a02f-4eaa-9573-e5f6dab8b4df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:39.183741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b24eb-a02f-4eaa-9573-e5f6dab8b4df service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:39.193224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1068/2146] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:00:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:00:40.717248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e436e18-a6d5-4440-80d8-1873924e5a62 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:40.828593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76782a-3700-4cd1-afe9-2a2093e4470b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:40.829762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76782a-3700-4cd1-afe9-2a2093e4470b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:40.830402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76782a-3700-4cd1-afe9-2a2093e4470b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:40.844136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e436e18-a6d5-4440-80d8-1873924e5a62 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:40.849067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:40.850867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:40.850867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:40.850867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] PATH_INFO: `/v3/users/21522dde4b5b471eb554d9b955716fec` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:40.852289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21522dde4b5b471eb554d9b955716fec)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:40.854345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:40.856907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ac76782a-3700-4cd1-afe9-2a2093e4470b None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:40.858358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ac76782a-3700-4cd1-afe9-2a2093e4470b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:40.866508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1069/2147] 10.222.0.94 () {58 vars in 967 bytes} [Fri Aug 26 16:00:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:40.881806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76782a-3700-4cd1-afe9-2a2093e4470b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:40.899117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76782a-3700-4cd1-afe9-2a2093e4470b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:40.902052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7743a3-b21e-4c71-9f26-cc45b504749d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:40.903660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7743a3-b21e-4c71-9f26-cc45b504749d service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:40.904055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7743a3-b21e-4c71-9f26-cc45b504749d service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:40.904330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7743a3-b21e-4c71-9f26-cc45b504749d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:40.913757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7743a3-b21e-4c71-9f26-cc45b504749d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:40.915226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7743a3-b21e-4c71-9f26-cc45b504749d service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:40.924970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1070/2148] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:00:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:00:41.216490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21522dde4b5b471eb554d9b955716fec'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:41.217162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21522dde4b5b471eb554d9b955716fec'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:41.231357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21522dde4b5b471eb554d9b955716fec'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:41.328524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21522dde4b5b471eb554d9b955716fec'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:41.337132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21522dde4b5b471eb554d9b955716fec'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:41.347380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1079/2149] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:00:40 2022] DELETE /identity/v3/users/21522dde4b5b471eb554d9b955716fec => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:41.351920 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81999fe-82eb-4d42-894c-1ec40b8702f2 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:41.352206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81999fe-82eb-4d42-894c-1ec40b8702f2 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:41.352481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81999fe-82eb-4d42-894c-1ec40b8702f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:41.373540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d81999fe-82eb-4d42-894c-1ec40b8702f2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:41.375029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d81999fe-82eb-4d42-894c-1ec40b8702f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:41.443685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1071/2150] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:41.454790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:41.457528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:41.533762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8081b6-bf32-49d7-b519-fc2eda9dcdf6 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:41.537815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:41.538476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:41.538756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:41.539063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:41.548182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:41.550622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:41.552127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:41.562497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1080/2151] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:42.051052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81999fe-82eb-4d42-894c-1ec40b8702f2 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:42.052579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d81999fe-82eb-4d42-894c-1ec40b8702f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:42.065729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81999fe-82eb-4d42-894c-1ec40b8702f2 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:42.080721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:42.081385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.081657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.081918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] PATH_INFO: `/v3/projects/1429c284e8e748058de2dae09d68680e` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.083230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1429c284e8e748058de2dae09d68680e)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:42.085080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:42.319467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1429c284e8e748058de2dae09d68680e', 'request_id': 'req-335b88f5-2476-468b-9c5f-6054a1429f6f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.325732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1429c284e8e748058de2dae09d68680e', 'request_id': 'req-335b88f5-2476-468b-9c5f-6054a1429f6f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.379914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:42.381513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:42.394339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ae439-7b58-4e62-950e-bc3e9af8d4c7 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:42.449281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1429c284e8e748058de2dae09d68680e', 'request_id': 'req-335b88f5-2476-468b-9c5f-6054a1429f6f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.458031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:42.458682 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.459001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.459272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] PATH_INFO: `/v3/users/796e6d2c1d3a4ef59c464fb2e61affe5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.460441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=796e6d2c1d3a4ef59c464fb2e61affe5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:42.463139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:42.465201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] The token cache is being invalidate because project 1429c284e8e748058de2dae09d68680e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:00:42.465726 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.466617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-335b88f5-2476-468b-9c5f-6054a1429f6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.468793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1072/2152] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:00:42 2022] DELETE /identity/v3/projects/1429c284e8e748058de2dae09d68680e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:42.473230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f157b48-04f2-4705-95f8-1cc45607a3c3 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.473518 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f157b48-04f2-4705-95f8-1cc45607a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.473832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f157b48-04f2-4705-95f8-1cc45607a3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.505524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8f157b48-04f2-4705-95f8-1cc45607a3c3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:42.508352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8f157b48-04f2-4705-95f8-1cc45607a3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:42.519610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1073/2153] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:00:42.533351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a41d-6914-48cc-9e63-10f922d18e3b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.533710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a41d-6914-48cc-9e63-10f922d18e3b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.539443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a41d-6914-48cc-9e63-10f922d18e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.545356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:42.556847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-87a4a41d-6914-48cc-9e63-10f922d18e3b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:42.558078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-87a4a41d-6914-48cc-9e63-10f922d18e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:42.567886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1074/2154] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:42.576012 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4a41d-6914-48cc-9e63-10f922d18e3b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:42.592461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4a41d-6914-48cc-9e63-10f922d18e3b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:42.606667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:42.607325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.607598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.609347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.609347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:00:42.609347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:42.610490 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:42.627573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1075/2155] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:00:42.631896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:42.647583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39730ec-027b-4dd9-9de1-500ff0615393 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:42.655413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fab0f8-e094-49d0-a2c9-9e0a14093b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:42.656239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fab0f8-e094-49d0-a2c9-9e0a14093b32 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.656673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fab0f8-e094-49d0-a2c9-9e0a14093b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.657114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fab0f8-e094-49d0-a2c9-9e0a14093b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.658289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fab0f8-e094-49d0-a2c9-9e0a14093b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:42.661487 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fab0f8-e094-49d0-a2c9-9e0a14093b32 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:42.738403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796e6d2c1d3a4ef59c464fb2e61affe5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.739175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796e6d2c1d3a4ef59c464fb2e61affe5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.752795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796e6d2c1d3a4ef59c464fb2e61affe5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.810362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1076/2156] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:00:42 2022] POST /identity/v3/projects => generated 391 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:00:42.814443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fab0f8-e094-49d0-a2c9-9e0a14093b32 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:42.847308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796e6d2c1d3a4ef59c464fb2e61affe5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.855401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a0619fe-e618-48a2-9d14-c998320d80ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796e6d2c1d3a4ef59c464fb2e61affe5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:42.862579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1081/2157] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:00:42 2022] DELETE /identity/v3/users/796e6d2c1d3a4ef59c464fb2e61affe5 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:00:42.869018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324d04-1e98-491d-8b54-e59fe64767e6 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.869669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324d04-1e98-491d-8b54-e59fe64767e6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.870954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324d04-1e98-491d-8b54-e59fe64767e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.895618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b0324d04-1e98-491d-8b54-e59fe64767e6 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:42.896821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b0324d04-1e98-491d-8b54-e59fe64767e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:42.907379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fab0f8-e094-49d0-a2c9-9e0a14093b32 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:42.987438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1082/2158] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:42.991771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cad7d4-e08f-4a55-a561-14e398d163a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:42.992456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cad7d4-e08f-4a55-a561-14e398d163a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:42.992742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cad7d4-e08f-4a55-a561-14e398d163a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:42.993074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cad7d4-e08f-4a55-a561-14e398d163a2 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:42.993653 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cad7d4-e08f-4a55-a561-14e398d163a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:42.997468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cad7d4-e08f-4a55-a561-14e398d163a2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:42.997705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0324d04-1e98-491d-8b54-e59fe64767e6 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:43.000705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b0324d04-1e98-491d-8b54-e59fe64767e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:43.029398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:43.071187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0324d04-1e98-491d-8b54-e59fe64767e6 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:43.075641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:43.076264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:43.076537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:43.076807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:43.087829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:43.089271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:43.099931 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1083/2159] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:00:43.162951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1077/2160] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:00:42 2022] POST /identity/v3/users => generated 296 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:00:43.164472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:43.165913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:43.182035 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad367f-681c-4712-a690-3a1f1214143f service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:43.201020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996bfa9-e5d7-4bef-9e78-326313421cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:43.201808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996bfa9-e5d7-4bef-9e78-326313421cff admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:43.212404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996bfa9-e5d7-4bef-9e78-326313421cff admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:43.212404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996bfa9-e5d7-4bef-9e78-326313421cff admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:43.212404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996bfa9-e5d7-4bef-9e78-326313421cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:43.212404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996bfa9-e5d7-4bef-9e78-326313421cff admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:43.219165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1084/2161] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:00:43.220453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cad7d4-e08f-4a55-a561-14e398d163a2 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:43.231185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cad7d4-e08f-4a55-a561-14e398d163a2 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:43.237648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:43.238263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:43.238553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:43.238825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:43.239517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:43.241189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:43.253892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1078/2162] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:00:43.274099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996bfa9-e5d7-4bef-9e78-326313421cff admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:43.298361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996bfa9-e5d7-4bef-9e78-326313421cff admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:43.308162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57905f89-01d2-4c47-ac38-746f85780dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:43.315581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57905f89-01d2-4c47-ac38-746f85780dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:43.315581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57905f89-01d2-4c47-ac38-746f85780dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:43.315581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57905f89-01d2-4c47-ac38-746f85780dd3 admin admin] PATH_INFO: `/v3/projects/2117707e65c348e3b1b19f05bc9138d1/users/ca92a39c395b493fb542664eb0da87c9/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:43.335448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57905f89-01d2-4c47-ac38-746f85780dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2117707e65c348e3b1b19f05bc9138d1, user_id=ca92a39c395b493fb542664eb0da87c9, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:43.338127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57905f89-01d2-4c47-ac38-746f85780dd3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:43.519992 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1085/2163] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:00:43 2022] PUT /identity/v3/projects/2117707e65c348e3b1b19f05bc9138d1/users/ca92a39c395b493fb542664eb0da87c9/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:43.531232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:43.539770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba28a5-3bd2-47a5-9b3b-d4186efce088 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:43.539975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:43.540095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba28a5-3bd2-47a5-9b3b-d4186efce088 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:43.540313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba28a5-3bd2-47a5-9b3b-d4186efce088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:43.566174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec320-fda7-4668-91ef-d44fec496786 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:43.602399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e4ba28a5-3bd2-47a5-9b3b-d4186efce088 None None] MFA Rules not processed for user `ca92a39c395b493fb542664eb0da87c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:43.602399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e4ba28a5-3bd2-47a5-9b3b-d4186efce088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:43.632394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:43.647499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:43.648202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:43.648478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:43.648862 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] PATH_INFO: `/v3/projects/aa98d1bd1c354b38b99eb85b741e44ab` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:43.650039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa98d1bd1c354b38b99eb85b741e44ab)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:43.652207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:43.663742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1086/2164] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:43.675332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba28a5-3bd2-47a5-9b3b-d4186efce088 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:43.676749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e4ba28a5-3bd2-47a5-9b3b-d4186efce088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:43.739987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba28a5-3bd2-47a5-9b3b-d4186efce088 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:43.743797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b6a92-e5dd-443a-979a-491c13ee74d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:43.744424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b6a92-e5dd-443a-979a-491c13ee74d8 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:43.744699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b6a92-e5dd-443a-979a-491c13ee74d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:43.744994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b6a92-e5dd-443a-979a-491c13ee74d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:43.755037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b6a92-e5dd-443a-979a-491c13ee74d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:43.756485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b6a92-e5dd-443a-979a-491c13ee74d8 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:43.766348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1087/2165] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:00:43.784848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa98d1bd1c354b38b99eb85b741e44ab', 'request_id': 'req-3d0c14b3-7b61-4b18-814a-be6129246793'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:43.791605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa98d1bd1c354b38b99eb85b741e44ab', 'request_id': 'req-3d0c14b3-7b61-4b18-814a-be6129246793'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:43.870500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa98d1bd1c354b38b99eb85b741e44ab', 'request_id': 'req-3d0c14b3-7b61-4b18-814a-be6129246793'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:43.889589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] The token cache is being invalidate because project aa98d1bd1c354b38b99eb85b741e44ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:00:43.889589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:43.889589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3d0c14b3-7b61-4b18-814a-be6129246793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:00:43.891381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1079/2166] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:00:43 2022] DELETE /identity/v3/projects/aa98d1bd1c354b38b99eb85b741e44ab => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:43.896973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e43091-4a20-400d-8f40-4cfde8e9b990 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:43.897260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e43091-4a20-400d-8f40-4cfde8e9b990 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:43.897532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e43091-4a20-400d-8f40-4cfde8e9b990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:43.924823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-94e43091-4a20-400d-8f40-4cfde8e9b990 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:43.926095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-94e43091-4a20-400d-8f40-4cfde8e9b990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:43.941845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:44.007047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1088/2167] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:44.015567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286507-522f-4bfe-87a9-f988ce382fba None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.015845 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286507-522f-4bfe-87a9-f988ce382fba None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.016211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4286507-522f-4bfe-87a9-f988ce382fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.037668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f4286507-522f-4bfe-87a9-f988ce382fba None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:44.040693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f4286507-522f-4bfe-87a9-f988ce382fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:44.066755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1080/2168] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:44.072448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e43091-4a20-400d-8f40-4cfde8e9b990 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:44.086969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e43091-4a20-400d-8f40-4cfde8e9b990 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:44.092339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:44.093132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.093461 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.093808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.095071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:00:44.095560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:44.097911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:44.115176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1089/2169] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:00:44.120441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4286507-522f-4bfe-87a9-f988ce382fba None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:44.129794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4286507-522f-4bfe-87a9-f988ce382fba None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:44.136027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b3e03-c9f0-4fe7-b2b5-0d498ae7675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:44.136720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3e03-c9f0-4fe7-b2b5-0d498ae7675c admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.137487 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3e03-c9f0-4fe7-b2b5-0d498ae7675c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.137866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3e03-c9f0-4fe7-b2b5-0d498ae7675c admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.138540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b3e03-c9f0-4fe7-b2b5-0d498ae7675c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:44.140773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b3e03-c9f0-4fe7-b2b5-0d498ae7675c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:44.248643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1081/2170] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:00:44 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:00:44.250618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:44.263041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88055a-81a8-4b8b-ad2b-38c901747a15 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:44.327594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde597-d8cb-4213-a529-7e7708127c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:44.328214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde597-d8cb-4213-a529-7e7708127c77 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.328708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde597-d8cb-4213-a529-7e7708127c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.328983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde597-d8cb-4213-a529-7e7708127c77 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.329615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fde597-d8cb-4213-a529-7e7708127c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:44.331744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fde597-d8cb-4213-a529-7e7708127c77 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:44.473349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1090/2171] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:00:44 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:00:44.476854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b3e03-c9f0-4fe7-b2b5-0d498ae7675c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:44.488278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b3e03-c9f0-4fe7-b2b5-0d498ae7675c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:44.492768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02859a-d777-408c-a3d3-323c184bfa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:44.493361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02859a-d777-408c-a3d3-323c184bfa5f admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.495466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02859a-d777-408c-a3d3-323c184bfa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.495466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02859a-d777-408c-a3d3-323c184bfa5f admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.495466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02859a-d777-408c-a3d3-323c184bfa5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:44.496844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02859a-d777-408c-a3d3-323c184bfa5f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:44.507384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1082/2172] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:00:44.515057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde597-d8cb-4213-a529-7e7708127c77 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:44.525970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde597-d8cb-4213-a529-7e7708127c77 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:44.531056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4e1c5-ef52-4474-bb16-4c712aa9b4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:44.531668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4e1c5-ef52-4474-bb16-4c712aa9b4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.531948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4e1c5-ef52-4474-bb16-4c712aa9b4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.532221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4e1c5-ef52-4474-bb16-4c712aa9b4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.532863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4e1c5-ef52-4474-bb16-4c712aa9b4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:44.534468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4e1c5-ef52-4474-bb16-4c712aa9b4d7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:44.546420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1091/2173] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:00:44.550686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02859a-d777-408c-a3d3-323c184bfa5f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:44.562451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02859a-d777-408c-a3d3-323c184bfa5f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:44.567487 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:44.568088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.568363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.570131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] PATH_INFO: `/v3/projects/ea82b722416a415393ff78428a277a01/users/3b3abcfcc7c6408994e27b7c9937c2be/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.587232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea82b722416a415393ff78428a277a01, user_id=3b3abcfcc7c6408994e27b7c9937c2be, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:44.590037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:44.712578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1083/2174] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:00:44 2022] PUT /identity/v3/projects/ea82b722416a415393ff78428a277a01/users/3b3abcfcc7c6408994e27b7c9937c2be/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:00:44.717695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b54d1-7434-4fe8-b9d5-90fad9168b51 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.718030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b54d1-7434-4fe8-b9d5-90fad9168b51 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.718294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b54d1-7434-4fe8-b9d5-90fad9168b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.745364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-172b54d1-7434-4fe8-b9d5-90fad9168b51 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:44.747679 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-172b54d1-7434-4fe8-b9d5-90fad9168b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:44.816361 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1092/2175] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:44.826547 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:44.828317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:44.891717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3d81b-215e-4324-8131-cb65711cf190 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:44.895475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ddbf5-7755-4e52-87b1-b3f78edad9e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:44.896088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ddbf5-7755-4e52-87b1-b3f78edad9e0 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:44.896362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ddbf5-7755-4e52-87b1-b3f78edad9e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:44.896630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ddbf5-7755-4e52-87b1-b3f78edad9e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:44.908289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ddbf5-7755-4e52-87b1-b3f78edad9e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:44.910056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ddbf5-7755-4e52-87b1-b3f78edad9e0 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:44.921313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1084/2176] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:00:52.054304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd83e8-7950-416e-bdad-852c14289fda None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:52.054872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd83e8-7950-416e-bdad-852c14289fda None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:52.054872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd83e8-7950-416e-bdad-852c14289fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:52.092309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5bfd83e8-7950-416e-bdad-852c14289fda None None] MFA Rules not processed for user `3b3abcfcc7c6408994e27b7c9937c2be`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:52.093637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5bfd83e8-7950-416e-bdad-852c14289fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:52.103918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:52.163963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1093/2177] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:52.169228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ddbf5-7755-4e52-87b1-b3f78edad9e0 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:52.179575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ddbf5-7755-4e52-87b1-b3f78edad9e0 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:52.184666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762f19b-3110-416e-8c03-6b30e53dc0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:52.185283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762f19b-3110-416e-8c03-6b30e53dc0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:52.185566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762f19b-3110-416e-8c03-6b30e53dc0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:52.185839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762f19b-3110-416e-8c03-6b30e53dc0d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:52.186529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762f19b-3110-416e-8c03-6b30e53dc0d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:52.188544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762f19b-3110-416e-8c03-6b30e53dc0d5 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:52.205913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:52.310217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1085/2178] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:00:52 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:00:52.310391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd83e8-7950-416e-bdad-852c14289fda None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:52.311831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd83e8-7950-416e-bdad-852c14289fda None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:52.376209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27643279-b972-4d4b-8149-eb31a8c1c886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:52.377511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27643279-b972-4d4b-8149-eb31a8c1c886 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:52.377854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27643279-b972-4d4b-8149-eb31a8c1c886 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:52.378348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27643279-b972-4d4b-8149-eb31a8c1c886 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:52.379402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27643279-b972-4d4b-8149-eb31a8c1c886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:52.382496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27643279-b972-4d4b-8149-eb31a8c1c886 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:52.482312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1094/2179] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:00:52 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:00:52.486288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762f19b-3110-416e-8c03-6b30e53dc0d5 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:52.500586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762f19b-3110-416e-8c03-6b30e53dc0d5 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:52.508193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f34ea-79f3-4e5a-be2f-f83a8b144335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:52.509071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f34ea-79f3-4e5a-be2f-f83a8b144335 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:52.509373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f34ea-79f3-4e5a-be2f-f83a8b144335 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:52.509683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f34ea-79f3-4e5a-be2f-f83a8b144335 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:52.510384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f34ea-79f3-4e5a-be2f-f83a8b144335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:52.513007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f34ea-79f3-4e5a-be2f-f83a8b144335 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:52.524597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1086/2180] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:00:52.530198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27643279-b972-4d4b-8149-eb31a8c1c886 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:52.541591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27643279-b972-4d4b-8149-eb31a8c1c886 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:52.546447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796f0f5-2975-4fd3-bd36-ecf687cde824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:52.547153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796f0f5-2975-4fd3-bd36-ecf687cde824 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:52.547425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796f0f5-2975-4fd3-bd36-ecf687cde824 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:52.547692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796f0f5-2975-4fd3-bd36-ecf687cde824 admin admin] PATH_INFO: `/v3/projects/46893205b06941b0b0123571e6a67a52/users/3f0ca12d53eb4c6b989c501a714ce2cb/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:52.565102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796f0f5-2975-4fd3-bd36-ecf687cde824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46893205b06941b0b0123571e6a67a52, user_id=3f0ca12d53eb4c6b989c501a714ce2cb, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:52.567835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796f0f5-2975-4fd3-bd36-ecf687cde824 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:52.643380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1095/2181] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:00:52 2022] PUT /identity/v3/projects/46893205b06941b0b0123571e6a67a52/users/3f0ca12d53eb4c6b989c501a714ce2cb/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:00:52.647591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f34ea-79f3-4e5a-be2f-f83a8b144335 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:52.658245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f34ea-79f3-4e5a-be2f-f83a8b144335 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:52.728780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a5670-47e8-4593-90db-376ee1407a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:52.729396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5670-47e8-4593-90db-376ee1407a7c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:52.729671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5670-47e8-4593-90db-376ee1407a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:52.730440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5670-47e8-4593-90db-376ee1407a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:52.731382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a5670-47e8-4593-90db-376ee1407a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:52.733039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a5670-47e8-4593-90db-376ee1407a7c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:52.743559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1087/2182] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:00:52.748599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796f0f5-2975-4fd3-bd36-ecf687cde824 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:52.759073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796f0f5-2975-4fd3-bd36-ecf687cde824 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:52.765725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:52.766329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:52.766601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:52.766876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] PATH_INFO: `/v3/projects/46893205b06941b0b0123571e6a67a52/users/3f0ca12d53eb4c6b989c501a714ce2cb/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:52.769104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46893205b06941b0b0123571e6a67a52, user_id=3f0ca12d53eb4c6b989c501a714ce2cb, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:52.772114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:52.785029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1096/2183] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:00:52 2022] PUT /identity/v3/projects/46893205b06941b0b0123571e6a67a52/users/3f0ca12d53eb4c6b989c501a714ce2cb/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:00:59.313192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ac49c-6e7a-4a2c-93d9-8727edf5ce24 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:59.313192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ac49c-6e7a-4a2c-93d9-8727edf5ce24 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:59.313639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ac49c-6e7a-4a2c-93d9-8727edf5ce24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:59.367742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3d9ac49c-6e7a-4a2c-93d9-8727edf5ce24 None None] MFA Rules not processed for user `3f0ca12d53eb4c6b989c501a714ce2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:00:59.369869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3d9ac49c-6e7a-4a2c-93d9-8727edf5ce24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:59.504424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1088/2184] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:00:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:00:59.523048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:00:59.525420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:00:59.633889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:00:59.641041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab2895-f253-4bbd-b72b-e58173077650 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:00:59.647121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:00:59.647815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:00:59.648132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:00:59.648360 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:00:59.723994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:00:59.725476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:00:59.736315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1097/2185] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:00:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:03.283886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741520a-c06e-491f-b6d4-3ca09d5bf733 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:03.286306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741520a-c06e-491f-b6d4-3ca09d5bf733 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:03.286306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741520a-c06e-491f-b6d4-3ca09d5bf733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:03.306373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d741520a-c06e-491f-b6d4-3ca09d5bf733 None None] MFA Rules not processed for user `3b3abcfcc7c6408994e27b7c9937c2be`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:03.308545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d741520a-c06e-491f-b6d4-3ca09d5bf733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:03.316342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1089/2186] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:09.868425 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=27 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:13.301305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=73 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:13.557606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:13.559123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:13.638963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b5ff1-e6e5-440c-bfd0-22d797fbc02d service nova] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:13.644235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:13.644875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:13.645258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:13.645448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] PATH_INFO: `/v3/users/ca92a39c395b493fb542664eb0da87c9` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:13.646659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca92a39c395b493fb542664eb0da87c9)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:13.649068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:13.878235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca92a39c395b493fb542664eb0da87c9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:13.878982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca92a39c395b493fb542664eb0da87c9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:13.892000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca92a39c395b493fb542664eb0da87c9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:13.972429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca92a39c395b493fb542664eb0da87c9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:13.981281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca92a39c395b493fb542664eb0da87c9'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:13.987498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1098/2187] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:01:13 2022] DELETE /identity/v3/users/ca92a39c395b493fb542664eb0da87c9 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:01:13.992124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41304a6-32b9-4236-b273-3ba6598e5c97 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:13.992358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41304a6-32b9-4236-b273-3ba6598e5c97 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:13.992889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41304a6-32b9-4236-b273-3ba6598e5c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:14.015877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-a41304a6-32b9-4236-b273-3ba6598e5c97 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:14.017133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a41304a6-32b9-4236-b273-3ba6598e5c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:14.085647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1090/2188] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:14.096350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:14.097985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:14.159614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a65703-e986-4cf5-aaae-45ea5eb9d272 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:14.163239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad7417-51c1-4c8d-ab93-9efa640dbda7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:14.163855 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad7417-51c1-4c8d-ab93-9efa640dbda7 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:14.164174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad7417-51c1-4c8d-ab93-9efa640dbda7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:14.164400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad7417-51c1-4c8d-ab93-9efa640dbda7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:14.174050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad7417-51c1-4c8d-ab93-9efa640dbda7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:14.175597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad7417-51c1-4c8d-ab93-9efa640dbda7 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:14.185321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1099/2189] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:14.662693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41304a6-32b9-4236-b273-3ba6598e5c97 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:14.667506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a41304a6-32b9-4236-b273-3ba6598e5c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:14.676154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41304a6-32b9-4236-b273-3ba6598e5c97 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:14.681994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:14.683172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:14.683172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:14.683661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] PATH_INFO: `/v3/projects/2117707e65c348e3b1b19f05bc9138d1` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:14.685538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] RBAC: Authorizing `identity:delete_project(project_id=2117707e65c348e3b1b19f05bc9138d1)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:14.688031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:14.821320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2117707e65c348e3b1b19f05bc9138d1', 'request_id': 'req-bf7dfc58-a788-460f-9744-1ed344adfbff'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:14.832081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2117707e65c348e3b1b19f05bc9138d1', 'request_id': 'req-bf7dfc58-a788-460f-9744-1ed344adfbff'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:14.949479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2117707e65c348e3b1b19f05bc9138d1', 'request_id': 'req-bf7dfc58-a788-460f-9744-1ed344adfbff'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:14.961936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] The token cache is being invalidate because project 2117707e65c348e3b1b19f05bc9138d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:01:14.962222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:14.962859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-bf7dfc58-a788-460f-9744-1ed344adfbff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:01:14.964453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1091/2190] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:01:14 2022] DELETE /identity/v3/projects/2117707e65c348e3b1b19f05bc9138d1 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:14.969590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fdedc-4791-490b-8e75-423f0bcf6bf1 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:14.970071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fdedc-4791-490b-8e75-423f0bcf6bf1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:14.970165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fdedc-4791-490b-8e75-423f0bcf6bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:14.991808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8c6fdedc-4791-490b-8e75-423f0bcf6bf1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:14.993014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8c6fdedc-4791-490b-8e75-423f0bcf6bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:15.011868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:15.057063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1100/2191] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:15.066575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ad709-aeab-42de-ad44-df6eb2f63b13 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.066879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ad709-aeab-42de-ad44-df6eb2f63b13 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.068557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ad709-aeab-42de-ad44-df6eb2f63b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.092204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-cb7ad709-aeab-42de-ad44-df6eb2f63b13 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:15.093515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-cb7ad709-aeab-42de-ad44-df6eb2f63b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:15.104088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1092/2192] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:15.109402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fdedc-4791-490b-8e75-423f0bcf6bf1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:15.119453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fdedc-4791-490b-8e75-423f0bcf6bf1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:15.124923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:15.125531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.125804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.126069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.126713 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:01:15.127044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:15.128640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:15.142350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1101/2193] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:01:15.146998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ad709-aeab-42de-ad44-df6eb2f63b13 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:15.157017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ad709-aeab-42de-ad44-df6eb2f63b13 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:15.162293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0be23-de2c-498c-8692-a873bb4f1540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:15.162956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0be23-de2c-498c-8692-a873bb4f1540 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.163243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0be23-de2c-498c-8692-a873bb4f1540 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.163509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0be23-de2c-498c-8692-a873bb4f1540 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.164144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0be23-de2c-498c-8692-a873bb4f1540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:15.166044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0be23-de2c-498c-8692-a873bb4f1540 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:15.272907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1093/2194] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:01:15 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:01:15.278230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:15.289034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db37ea-b6f2-49af-9526-fbd76d7ee911 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:15.350827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6222cd-7fe2-4514-abb5-2afab787706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:15.351530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6222cd-7fe2-4514-abb5-2afab787706b admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.351816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6222cd-7fe2-4514-abb5-2afab787706b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.352088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6222cd-7fe2-4514-abb5-2afab787706b admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.352731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6222cd-7fe2-4514-abb5-2afab787706b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:15.355585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6222cd-7fe2-4514-abb5-2afab787706b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:15.464616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1102/2195] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:01:15 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:15.468770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0be23-de2c-498c-8692-a873bb4f1540 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:15.479594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0be23-de2c-498c-8692-a873bb4f1540 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:15.485257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa1317-16a5-404f-bd44-d16aab601798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:15.485872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa1317-16a5-404f-bd44-d16aab601798 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.486174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa1317-16a5-404f-bd44-d16aab601798 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.486462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa1317-16a5-404f-bd44-d16aab601798 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.487157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa1317-16a5-404f-bd44-d16aab601798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:15.488766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa1317-16a5-404f-bd44-d16aab601798 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:15.500521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1094/2196] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:15.504532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6222cd-7fe2-4514-abb5-2afab787706b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:15.515288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6222cd-7fe2-4514-abb5-2afab787706b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:15.520344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875c71e-35c8-4e57-b0e5-7776228d8c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:15.520960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c71e-35c8-4e57-b0e5-7776228d8c8c admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.521234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c71e-35c8-4e57-b0e5-7776228d8c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.521500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c71e-35c8-4e57-b0e5-7776228d8c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.522152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875c71e-35c8-4e57-b0e5-7776228d8c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:15.523808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875c71e-35c8-4e57-b0e5-7776228d8c8c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:15.538983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1103/2197] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:01:15.540417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa1317-16a5-404f-bd44-d16aab601798 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:15.553630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa1317-16a5-404f-bd44-d16aab601798 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:15.558180 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:15.558820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.559222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.559498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] PATH_INFO: `/v3/projects/070e0f497888488d863e2f63b87a31ff/users/36d61fd7e97d433e886aaf495f703751/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.578391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=070e0f497888488d863e2f63b87a31ff, user_id=36d61fd7e97d433e886aaf495f703751, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:15.581526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:15.692727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1095/2198] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:15 2022] PUT /identity/v3/projects/070e0f497888488d863e2f63b87a31ff/users/36d61fd7e97d433e886aaf495f703751/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:15.698449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5883b63-94ce-4902-a84d-70e5d2e81f73 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.698739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5883b63-94ce-4902-a84d-70e5d2e81f73 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.699042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5883b63-94ce-4902-a84d-70e5d2e81f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.719759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b5883b63-94ce-4902-a84d-70e5d2e81f73 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:15.721190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b5883b63-94ce-4902-a84d-70e5d2e81f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:15.784814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1104/2199] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:15.794878 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:15.797072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:15.821239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:15.868205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43eb80-443b-4a55-9d58-b53fa002b965 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:15.871836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9859567-23f7-4885-b87f-6becbe23f1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:15.876781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9859567-23f7-4885-b87f-6becbe23f1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:15.876781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9859567-23f7-4885-b87f-6becbe23f1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:15.876781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9859567-23f7-4885-b87f-6becbe23f1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:15.885789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9859567-23f7-4885-b87f-6becbe23f1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:15.887284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9859567-23f7-4885-b87f-6becbe23f1d4 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:15.898070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1096/2200] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:23.280354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9ca8-cba3-4a60-ac0c-98524c951148 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:23.280901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9ca8-cba3-4a60-ac0c-98524c951148 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:23.280901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9ca8-cba3-4a60-ac0c-98524c951148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:23.311637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ab7f9ca8-cba3-4a60-ac0c-98524c951148 None None] MFA Rules not processed for user `36d61fd7e97d433e886aaf495f703751`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:23.312863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ab7f9ca8-cba3-4a60-ac0c-98524c951148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:23.330027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:23.385944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1105/2201] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:23.386871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9859567-23f7-4885-b87f-6becbe23f1d4 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:23.403142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9859567-23f7-4885-b87f-6becbe23f1d4 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:23.417700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c75475-cebb-4fcb-9814-b38a7ba50eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:23.417700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c75475-cebb-4fcb-9814-b38a7ba50eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:23.417700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c75475-cebb-4fcb-9814-b38a7ba50eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:23.417700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c75475-cebb-4fcb-9814-b38a7ba50eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:23.418406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c75475-cebb-4fcb-9814-b38a7ba50eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:23.418406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c75475-cebb-4fcb-9814-b38a7ba50eb0 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:23.626986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1097/2202] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:01:23 2022] POST /identity/v3/projects => generated 385 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:23.644167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9ca8-cba3-4a60-ac0c-98524c951148 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:23.656062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9ca8-cba3-4a60-ac0c-98524c951148 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:23.714230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933822ea-4b0a-41f8-bdd9-d0d8bbe21204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:23.714872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933822ea-4b0a-41f8-bdd9-d0d8bbe21204 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:23.715199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933822ea-4b0a-41f8-bdd9-d0d8bbe21204 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:23.715485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933822ea-4b0a-41f8-bdd9-d0d8bbe21204 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:23.716111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933822ea-4b0a-41f8-bdd9-d0d8bbe21204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:23.719037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933822ea-4b0a-41f8-bdd9-d0d8bbe21204 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:23.849307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1106/2203] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:01:23 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:01:23.853817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c75475-cebb-4fcb-9814-b38a7ba50eb0 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:23.869175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c75475-cebb-4fcb-9814-b38a7ba50eb0 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:23.876003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4e82-07e0-437f-892e-4838d16b32bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:23.876650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4e82-07e0-437f-892e-4838d16b32bf admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:23.877293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4e82-07e0-437f-892e-4838d16b32bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:23.877568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4e82-07e0-437f-892e-4838d16b32bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:23.878220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e4e82-07e0-437f-892e-4838d16b32bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:23.879989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e4e82-07e0-437f-892e-4838d16b32bf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:23.896496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1098/2204] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:23.897693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933822ea-4b0a-41f8-bdd9-d0d8bbe21204 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:23.910692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933822ea-4b0a-41f8-bdd9-d0d8bbe21204 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:23.916255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31771-c1c2-40ce-a2ae-7350f766384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:23.916855 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31771-c1c2-40ce-a2ae-7350f766384f admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:23.917132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31771-c1c2-40ce-a2ae-7350f766384f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:23.917426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c31771-c1c2-40ce-a2ae-7350f766384f admin admin] PATH_INFO: `/v3/projects/2e294fd9a4bd4fea914485eec964748b/users/ee66a17a796c4d0ebfbc1d5779c97450/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:23.934584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c31771-c1c2-40ce-a2ae-7350f766384f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e294fd9a4bd4fea914485eec964748b, user_id=ee66a17a796c4d0ebfbc1d5779c97450, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:23.938302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c31771-c1c2-40ce-a2ae-7350f766384f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:24.045896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1107/2205] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:23 2022] PUT /identity/v3/projects/2e294fd9a4bd4fea914485eec964748b/users/ee66a17a796c4d0ebfbc1d5779c97450/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:24.047299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4e82-07e0-437f-892e-4838d16b32bf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:24.058794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4e82-07e0-437f-892e-4838d16b32bf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:24.126274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328eaf4-6e3f-4e9b-bc26-25e1a9c06fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:24.126892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328eaf4-6e3f-4e9b-bc26-25e1a9c06fce admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:24.127200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328eaf4-6e3f-4e9b-bc26-25e1a9c06fce admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:24.127509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328eaf4-6e3f-4e9b-bc26-25e1a9c06fce admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:24.128149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328eaf4-6e3f-4e9b-bc26-25e1a9c06fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:24.129827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328eaf4-6e3f-4e9b-bc26-25e1a9c06fce admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:24.140542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1099/2206] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:01:24.144825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31771-c1c2-40ce-a2ae-7350f766384f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:24.156372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c31771-c1c2-40ce-a2ae-7350f766384f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:24.163294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:24.164279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:24.164778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:24.165254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] PATH_INFO: `/v3/projects/2e294fd9a4bd4fea914485eec964748b/users/ee66a17a796c4d0ebfbc1d5779c97450/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:24.167248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e294fd9a4bd4fea914485eec964748b, user_id=ee66a17a796c4d0ebfbc1d5779c97450, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:24.170034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:24.181562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1108/2207] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:01:24 2022] PUT /identity/v3/projects/2e294fd9a4bd4fea914485eec964748b/users/ee66a17a796c4d0ebfbc1d5779c97450/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:01:31.933609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942e857-4880-4086-b176-09ad135efd2c None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:31.934120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942e857-4880-4086-b176-09ad135efd2c None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:31.934203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942e857-4880-4086-b176-09ad135efd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:31.965869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-1942e857-4880-4086-b176-09ad135efd2c None None] MFA Rules not processed for user `ee66a17a796c4d0ebfbc1d5779c97450`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:31.967714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1942e857-4880-4086-b176-09ad135efd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:32.114144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1100/2208] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:32.127542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:32.129259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:32.224038 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3564a-b043-438a-8c68-e7126f262bf1 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:32.227813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23ddf8f600454da7ae9b715cd4cbd186', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:32.228295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:32.228574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:32.228846 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:32.254560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:32.288406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:32.289797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:32.299189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1109/2209] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:32.713788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9296931-3ad0-4232-bcc4-513046c04740 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:32.714276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9296931-3ad0-4232-bcc4-513046c04740 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:32.714454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9296931-3ad0-4232-bcc4-513046c04740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:32.737951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-b9296931-3ad0-4232-bcc4-513046c04740 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:32.739508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b9296931-3ad0-4232-bcc4-513046c04740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:32.747102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:32.811740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1101/2210] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 16:01:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:32.824115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:32.825544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:32.903546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c2afc-07d3-48b8-bd30-dc3eb9c04f74 service glance] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:32.909222 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b62d1-1d70-4852-8ac0-d1e5f7ff141e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:32.909839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b62d1-1d70-4852-8ac0-d1e5f7ff141e service swift] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:32.910144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b62d1-1d70-4852-8ac0-d1e5f7ff141e service swift] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:32.911124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b62d1-1d70-4852-8ac0-d1e5f7ff141e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:32.920521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b62d1-1d70-4852-8ac0-d1e5f7ff141e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:32.922120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b62d1-1d70-4852-8ac0-d1e5f7ff141e service swift] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:32.935024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1110/2211] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:01:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 16:01:35.787543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84d45f-2064-4c64-82f0-d678f5ea287d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:35.788001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84d45f-2064-4c64-82f0-d678f5ea287d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:35.788083 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84d45f-2064-4c64-82f0-d678f5ea287d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:35.823112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-1a84d45f-2064-4c64-82f0-d678f5ea287d None None] MFA Rules not processed for user `b50041d36b0b430badd6442a2f09f063`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:35.824494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1a84d45f-2064-4c64-82f0-d678f5ea287d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:35.910966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1102/2212] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:35.956569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b62d1-1d70-4852-8ac0-d1e5f7ff141e service swift] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:35.969471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b62d1-1d70-4852-8ac0-d1e5f7ff141e service swift] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:35.974131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5790bf-df41-42bc-8875-75c051b3d098 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23ddf8f600454da7ae9b715cd4cbd186', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:35.974614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5790bf-df41-42bc-8875-75c051b3d098 service glance] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:35.974896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5790bf-df41-42bc-8875-75c051b3d098 service glance] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:35.975224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5790bf-df41-42bc-8875-75c051b3d098 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:35.979831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8d5790bf-df41-42bc-8875-75c051b3d098 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:36.081621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5790bf-df41-42bc-8875-75c051b3d098 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:36.083262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5790bf-df41-42bc-8875-75c051b3d098 service glance] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:36.101557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1111/2213] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:38.112070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84d45f-2064-4c64-82f0-d678f5ea287d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:38.244589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84d45f-2064-4c64-82f0-d678f5ea287d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:38.251255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:38.251890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:38.252168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:38.252444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:38.258530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:38.278636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:38.281813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:38.307284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1103/2214] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:40.007695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72197b-3c57-4026-ac08-4aafb03c8575 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:40.007695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72197b-3c57-4026-ac08-4aafb03c8575 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:40.007695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72197b-3c57-4026-ac08-4aafb03c8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:40.030529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cd72197b-3c57-4026-ac08-4aafb03c8575 None None] MFA Rules not processed for user `4940f8a9c15a483eab251df392789534`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:40.030529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cd72197b-3c57-4026-ac08-4aafb03c8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:40.082733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:01:40.117440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1112/2215] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:01:43.260932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:43.275225 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1054e9d-c2b8-4422-9b6f-58c7b8f806b1 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:43.282142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:43.283206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:43.285998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:43.285998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:43.285998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:43.319599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:43.321849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:43.334504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1104/2216] 10.222.0.94 () {60 vars in 1396 bytes} [Fri Aug 26 16:01:43 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:47.069323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df88e-ec23-4399-9686-200d667658cf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.069854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df88e-ec23-4399-9686-200d667658cf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.069854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df88e-ec23-4399-9686-200d667658cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.090493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-850df88e-ec23-4399-9686-200d667658cf None None] MFA Rules not processed for user `36d61fd7e97d433e886aaf495f703751`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:47.091975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-850df88e-ec23-4399-9686-200d667658cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:47.099247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1113/2217] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:01:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:47.238113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:47.239767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:47.255952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679445ca-5971-43e2-bd24-396de99c168f service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:47.261368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118028e-8027-432b-9529-551345e1eb3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:47.262045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118028e-8027-432b-9529-551345e1eb3b service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:47.262321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118028e-8027-432b-9529-551345e1eb3b service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:47.262587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118028e-8027-432b-9529-551345e1eb3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:47.264594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1118028e-8027-432b-9529-551345e1eb3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:47.275880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118028e-8027-432b-9529-551345e1eb3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:47.277503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118028e-8027-432b-9529-551345e1eb3b service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:47.288569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1105/2218] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:01:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:54.142714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850df88e-ec23-4399-9686-200d667658cf None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:54.152225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850df88e-ec23-4399-9686-200d667658cf None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:54.156525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:54.157069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:54.157341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:54.157605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:54.159457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:54.218209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:54.219678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:54.228647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1114/2219] 10.222.0.94 () {60 vars in 1396 bytes} [Fri Aug 26 16:01:54 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:01:55.230963 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fcfd-8735-4d46-8942-8190a3cc59bd None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.231499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fcfd-8735-4d46-8942-8190a3cc59bd None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.231606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fcfd-8735-4d46-8942-8190a3cc59bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.253155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6131fcfd-8735-4d46-8942-8190a3cc59bd None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:01:55.260977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6131fcfd-8735-4d46-8942-8190a3cc59bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:01:55.262366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1106/2220] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:01:55.275893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:01:55.287676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5274679-d81e-4397-995c-b95ac7a7c810 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:01:55.293219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5bbbe-b93d-4fa1-a649-ce348b0ed90b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:01:55.303670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5bbbe-b93d-4fa1-a649-ce348b0ed90b service swift] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:01:55.303670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5bbbe-b93d-4fa1-a649-ce348b0ed90b service swift] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:01:55.303670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5bbbe-b93d-4fa1-a649-ce348b0ed90b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:01:55.303670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5bbbe-b93d-4fa1-a649-ce348b0ed90b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:01:55.305677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5bbbe-b93d-4fa1-a649-ce348b0ed90b service swift] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:01:55.316227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1115/2221] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:01:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:02:00.513921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280b5a9-180e-4b73-bb18-d808dd04c1ca None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:00.514730 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280b5a9-180e-4b73-bb18-d808dd04c1ca None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:00.515200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280b5a9-180e-4b73-bb18-d808dd04c1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:00.540662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0280b5a9-180e-4b73-bb18-d808dd04c1ca None None] MFA Rules not processed for user `228837054cd648acb0df93dd423e1d83`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:00.542325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0280b5a9-180e-4b73-bb18-d808dd04c1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:00.554041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1107/2222] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:01.045122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5bbbe-b93d-4fa1-a649-ce348b0ed90b service swift] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:01.062477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5bbbe-b93d-4fa1-a649-ce348b0ed90b service swift] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:01.066328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:01.067197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.067642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.068078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.070141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:01.082449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:01.084362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:01.094615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1116/2223] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:01.567005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280b5a9-180e-4b73-bb18-d808dd04c1ca None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:01.568555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0280b5a9-180e-4b73-bb18-d808dd04c1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:01.613010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:01.676318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280b5a9-180e-4b73-bb18-d808dd04c1ca None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:01.715493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:01.716981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.717455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.717925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] PATH_INFO: `/v3/users/a0ecfd3d64ba4fef820a31b1522125ff` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.721530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ecfd3d64ba4fef820a31b1522125ff)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:01.723823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:01.754744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:01.757562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:01.925071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d754085-8f1b-4224-bf13-6e2f976719a9 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:01.932330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:01.933257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:01.933717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:01.934142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:01.937335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:01.962095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:01.963972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:01.981178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1117/2224] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:02.587722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ecfd3d64ba4fef820a31b1522125ff'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:02.587722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ecfd3d64ba4fef820a31b1522125ff'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:02.596181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ecfd3d64ba4fef820a31b1522125ff'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:02.596916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ecfd3d64ba4fef820a31b1522125ff'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:02.708879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ecfd3d64ba4fef820a31b1522125ff'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:02.725253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1108/2225] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:02:01 2022] DELETE /identity/v3/users/a0ecfd3d64ba4fef820a31b1522125ff => generated 0 bytes in 1158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:02:03.176683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:03.195145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64316275-8b45-46fd-9aa7-c1ce76bf79b5 service placement] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:03.290871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:03.329850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:03.332154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:03.332889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:03.334898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] PATH_INFO: `/v3/projects/f3a9b259e9d54603af734306adbfe16c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:03.336836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a9b259e9d54603af734306adbfe16c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:03.338787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:03.517948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a9b259e9d54603af734306adbfe16c', 'request_id': 'req-97ea3011-e46d-41be-b2f5-13a4e8577847'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:03.524012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a9b259e9d54603af734306adbfe16c', 'request_id': 'req-97ea3011-e46d-41be-b2f5-13a4e8577847'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:03.594401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a9b259e9d54603af734306adbfe16c', 'request_id': 'req-97ea3011-e46d-41be-b2f5-13a4e8577847'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:03.608838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] The token cache is being invalidate because project f3a9b259e9d54603af734306adbfe16c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:02:03.609161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:03.609856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:02:03.614974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1118/2226] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:02:03 2022] DELETE /identity/v3/projects/f3a9b259e9d54603af734306adbfe16c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:02:05.817867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:05.826349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:05.887667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b671f-e9cb-48a1-9a54-28fb528aac96 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:05.941217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:05.941951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:05.942356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:05.942681 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:05.944513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:05.954770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:05.956216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:05.965002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1109/2227] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:11.332803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:11.335054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:11.416770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3011-e46d-41be-b2f5-13a4e8577847 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:11.423741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:11.424441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:11.424791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:11.425127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:11.427177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:11.441656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:11.443810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:11.457019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1119/2228] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:14.328252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:14.330260 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:14.336247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:14.390685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58eff8a-3f70-41e8-80c6-6da08364ebf7 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:14.394277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:14.394874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:14.395180 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:14.395446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:14.397142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:14.458320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:14.459705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:14.469642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1110/2229] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:15.967244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:15.968745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:15.980309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40d4d8-ddb8-4475-9f41-4af26bdbeea7 service nova] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:15.984838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4cd6c-bf3f-4d61-aba0-27e771e5bea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:15.985449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4cd6c-bf3f-4d61-aba0-27e771e5bea8 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:15.985730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4cd6c-bf3f-4d61-aba0-27e771e5bea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:15.986001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4cd6c-bf3f-4d61-aba0-27e771e5bea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:15.988195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2ed4cd6c-bf3f-4d61-aba0-27e771e5bea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:16.046459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:16.059888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4cd6c-bf3f-4d61-aba0-27e771e5bea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:16.061401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4cd6c-bf3f-4d61-aba0-27e771e5bea8 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:16.072251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1120/2230] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:19.557404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:19.559853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:19.586192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf29fd3-e303-43ff-8c21-c0f114467f89 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:19.595385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:19.596332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:19.596816 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:19.597297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:19.600445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:19.616153 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:19.617863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:19.636006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1111/2231] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:26.159590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:27.247613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f1a46-5427-4cb3-88f3-6a1829640353 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:27.248201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f1a46-5427-4cb3-88f3-6a1829640353 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:27.248201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f1a46-5427-4cb3-88f3-6a1829640353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:27.268001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-264f1a46-5427-4cb3-88f3-6a1829640353 None None] MFA Rules not processed for user `7069a9b54a3245b3ab9671730344e6d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:02:27.269229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-264f1a46-5427-4cb3-88f3-6a1829640353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:27.276152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1121/2232] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:02:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:02:29.630393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=53 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:34.822155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:34.823669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:34.836294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969df39c-8b9f-4b1b-9edc-6fe35efc3e31 service nova] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:34.840089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:34.840696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:34.840968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:34.841239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:34.843297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:34.916579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:34.918058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:34.929722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1112/2233] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:37.261909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:39.803590 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f1a46-5427-4cb3-88f3-6a1829640353 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:02:39.816585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f1a46-5427-4cb3-88f3-6a1829640353 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:02:39.822611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:02:39.823258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:02:39.823537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:02:39.823806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:02:39.825671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:02:39.838780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:02:39.840410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:02:39.852209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1122/2234] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:02:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:02:44.929872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=30 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:02:49.847278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=6 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:01.265853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:01.275402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4eab-b0b9-4d89-aba6-1939f9738b0e service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:01.279010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:01.279602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:01.279877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:01.280172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:01.281884 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:01.347089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:01.348527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:01.358200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1113/2235] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:04.928967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:04.930406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:05.005337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973b5aa-e837-4f12-98cf-8d60745e061e service nova] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:05.010376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:05.011010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:05.011292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:05.011568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:05.013295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:05.024599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:05.026173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:05.036125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1123/2236] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:05.148771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:05.150287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:05.164072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5b625-7b3f-49ce-b74a-80fa184cbb3a service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:05.168834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:05.169469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:05.169751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:05.170023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] PATH_INFO: `/v3/users/4940f8a9c15a483eab251df392789534` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:05.171167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4940f8a9c15a483eab251df392789534)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:05.173109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:05.356347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4940f8a9c15a483eab251df392789534'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:05.357008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4940f8a9c15a483eab251df392789534'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:05.369250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4940f8a9c15a483eab251df392789534'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:05.375695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4940f8a9c15a483eab251df392789534'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:05.462379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-1d385798-bb32-41cb-960e-829d6786cf35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4940f8a9c15a483eab251df392789534'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:05.475621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1114/2237] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:05 2022] DELETE /identity/v3/users/4940f8a9c15a483eab251df392789534 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:05.749808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:05.760820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfa00d-5003-43bb-a8cd-0f7868d69b99 service placement] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:05.819337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:05.819970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:05.820249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:05.820551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] PATH_INFO: `/v3/projects/5f170813ecff47a68d7610de9d769287` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:05.821684 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f170813ecff47a68d7610de9d769287)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:05.823689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:05.933862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f170813ecff47a68d7610de9d769287', 'request_id': 'req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:05.939368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f170813ecff47a68d7610de9d769287', 'request_id': 'req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:06.010721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f170813ecff47a68d7610de9d769287', 'request_id': 'req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:06.023206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] The token cache is being invalidate because project 5f170813ecff47a68d7610de9d769287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:06.023493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:06.024130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cd00d2a-05a8-4d26-bbe4-a826e2b0ee2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:06.025652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1124/2238] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:05 2022] DELETE /identity/v3/projects/5f170813ecff47a68d7610de9d769287 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:03:06.030652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f0f63-9ee3-4b8a-8117-26dbcb9cc2d9 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.030960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f0f63-9ee3-4b8a-8117-26dbcb9cc2d9 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.031243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f0f63-9ee3-4b8a-8117-26dbcb9cc2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.050746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-191f0f63-9ee3-4b8a-8117-26dbcb9cc2d9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:06.051994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-191f0f63-9ee3-4b8a-8117-26dbcb9cc2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:06.111521 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1115/2239] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:06.119493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317cf8f-0029-4a6a-b694-2728957c232c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.119777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317cf8f-0029-4a6a-b694-2728957c232c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.120049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317cf8f-0029-4a6a-b694-2728957c232c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.140039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a317cf8f-0029-4a6a-b694-2728957c232c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:06.141239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a317cf8f-0029-4a6a-b694-2728957c232c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:06.149695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1125/2240] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:06.153993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f0f63-9ee3-4b8a-8117-26dbcb9cc2d9 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:06.164125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f0f63-9ee3-4b8a-8117-26dbcb9cc2d9 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:06.168689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:06.169290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.169565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.169834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.170475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:06.170774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:06.172420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:06.173898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:06.185405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1116/2241] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:03:06.189718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317cf8f-0029-4a6a-b694-2728957c232c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:06.200068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317cf8f-0029-4a6a-b694-2728957c232c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:06.204873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a61c3c-8db0-4924-968c-428f1f03dc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:06.205549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a61c3c-8db0-4924-968c-428f1f03dc52 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.205896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a61c3c-8db0-4924-968c-428f1f03dc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.206323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a61c3c-8db0-4924-968c-428f1f03dc52 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.207259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a61c3c-8db0-4924-968c-428f1f03dc52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:06.209249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a61c3c-8db0-4924-968c-428f1f03dc52 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:06.220655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:06.306279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1126/2242] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:06 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:06.310609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:06.322181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597fe8-3b09-4013-9f0d-212079c9addc admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:06.381047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e70d28-26ad-4206-bc21-8f8b4545524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:06.381678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e70d28-26ad-4206-bc21-8f8b4545524d admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.381953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e70d28-26ad-4206-bc21-8f8b4545524d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.382221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e70d28-26ad-4206-bc21-8f8b4545524d admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.382839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e70d28-26ad-4206-bc21-8f8b4545524d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:06.384788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e70d28-26ad-4206-bc21-8f8b4545524d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:06.511916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1117/2243] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:06 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:06.516056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a61c3c-8db0-4924-968c-428f1f03dc52 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:06.526376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a61c3c-8db0-4924-968c-428f1f03dc52 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:06.531449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b60b41-40dd-48f2-b9a1-b70fee5f730e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:06.532063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60b41-40dd-48f2-b9a1-b70fee5f730e admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.532324 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60b41-40dd-48f2-b9a1-b70fee5f730e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.532594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60b41-40dd-48f2-b9a1-b70fee5f730e admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.533276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b60b41-40dd-48f2-b9a1-b70fee5f730e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:06.535288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b60b41-40dd-48f2-b9a1-b70fee5f730e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:06.546720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1127/2244] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:06.549509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e70d28-26ad-4206-bc21-8f8b4545524d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:06.560467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e70d28-26ad-4206-bc21-8f8b4545524d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:06.565210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9b0de-2052-4308-8feb-5b9cb397595a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:06.566460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b0de-2052-4308-8feb-5b9cb397595a admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.566895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b0de-2052-4308-8feb-5b9cb397595a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.567406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b0de-2052-4308-8feb-5b9cb397595a admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.568131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9b0de-2052-4308-8feb-5b9cb397595a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:06.570010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9b0de-2052-4308-8feb-5b9cb397595a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:06.580417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1118/2245] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:06.586576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b60b41-40dd-48f2-b9a1-b70fee5f730e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:06.597836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b60b41-40dd-48f2-b9a1-b70fee5f730e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:06.602426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:06.603097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.603368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.603631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] PATH_INFO: `/v3/projects/4084946ec94b47e6afa4940a3e322b8e/users/f33b855df2fb4a9da806dd0602bf9bf1/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.618602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4084946ec94b47e6afa4940a3e322b8e, user_id=f33b855df2fb4a9da806dd0602bf9bf1, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:06.621256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:06.701858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1128/2246] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:06 2022] PUT /identity/v3/projects/4084946ec94b47e6afa4940a3e322b8e/users/f33b855df2fb4a9da806dd0602bf9bf1/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:06.713168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ea25-18f1-46be-99b8-8804436f7960 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.713446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ea25-18f1-46be-99b8-8804436f7960 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.718669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ea25-18f1-46be-99b8-8804436f7960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.742993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-eee5ea25-18f1-46be-99b8-8804436f7960 None None] MFA Rules not processed for user `f33b855df2fb4a9da806dd0602bf9bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:06.744226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-eee5ea25-18f1-46be-99b8-8804436f7960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:06.809943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1119/2247] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:06.822043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:06.823719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:06.898272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5f3e8-9c14-44ca-92eb-79f0af6620a2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:06.904122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b488d-9f9a-45e8-8529-48efa2727d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:06.904730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b488d-9f9a-45e8-8529-48efa2727d67 service nova] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:06.905005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b488d-9f9a-45e8-8529-48efa2727d67 service nova] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:06.905274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b488d-9f9a-45e8-8529-48efa2727d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:06.912984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b488d-9f9a-45e8-8529-48efa2727d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:06.918823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b488d-9f9a-45e8-8529-48efa2727d67 service nova] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:06.923341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1129/2248] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:16.453270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5ea25-18f1-46be-99b8-8804436f7960 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:16.517584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5ea25-18f1-46be-99b8-8804436f7960 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:16.522141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:16.523119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:16.523429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:16.523710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] PATH_INFO: `/v3/users/f33b855df2fb4a9da806dd0602bf9bf1` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:16.524852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f33b855df2fb4a9da806dd0602bf9bf1)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:16.526771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:16.529354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:16.744438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f33b855df2fb4a9da806dd0602bf9bf1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:16.745140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f33b855df2fb4a9da806dd0602bf9bf1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:16.757270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f33b855df2fb4a9da806dd0602bf9bf1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:16.765796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f33b855df2fb4a9da806dd0602bf9bf1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:16.867642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f33b855df2fb4a9da806dd0602bf9bf1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:16.880595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1120/2249] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:16 2022] DELETE /identity/v3/users/f33b855df2fb4a9da806dd0602bf9bf1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:16.885266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5eb8-acc3-44e2-bfa7-f572ecc2d226 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:16.885771 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5eb8-acc3-44e2-bfa7-f572ecc2d226 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:16.885871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5eb8-acc3-44e2-bfa7-f572ecc2d226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:16.905309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8a2b5eb8-acc3-44e2-bfa7-f572ecc2d226 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:16.906574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8a2b5eb8-acc3-44e2-bfa7-f572ecc2d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:16.967129 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1130/2250] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:16.977426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:16.978944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:17.038550 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132773ba-cb53-40e8-8da6-fbec94918a78 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:17.042520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb0579-251d-49af-9a9d-dd8186f41dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:17.043160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb0579-251d-49af-9a9d-dd8186f41dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:17.043440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb0579-251d-49af-9a9d-dd8186f41dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:17.043712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb0579-251d-49af-9a9d-dd8186f41dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:17.054722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb0579-251d-49af-9a9d-dd8186f41dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:17.057915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb0579-251d-49af-9a9d-dd8186f41dd2 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:17.070139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1121/2251] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:17.635777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5eb8-acc3-44e2-bfa7-f572ecc2d226 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:17.639604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8a2b5eb8-acc3-44e2-bfa7-f572ecc2d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:17.656130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5eb8-acc3-44e2-bfa7-f572ecc2d226 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:17.678432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:17.679402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:17.679702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:17.679973 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] PATH_INFO: `/v3/projects/4084946ec94b47e6afa4940a3e322b8e` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:17.681552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4084946ec94b47e6afa4940a3e322b8e)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:17.684381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:17.832833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:17.843292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4084946ec94b47e6afa4940a3e322b8e', 'request_id': 'req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:17.850139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4084946ec94b47e6afa4940a3e322b8e', 'request_id': 'req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:17.977018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4084946ec94b47e6afa4940a3e322b8e', 'request_id': 'req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:17.989094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] The token cache is being invalidate because project 4084946ec94b47e6afa4940a3e322b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:17.989439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:17.990071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f51c52d3-c34c-4e1b-a19f-38c6dfd85f46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:17.992155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1131/2252] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:17 2022] DELETE /identity/v3/projects/4084946ec94b47e6afa4940a3e322b8e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:17.997132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621dc18-0863-43ad-968c-f1b89acbc929 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:17.997622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621dc18-0863-43ad-968c-f1b89acbc929 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:17.997740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621dc18-0863-43ad-968c-f1b89acbc929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.018189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6621dc18-0863-43ad-968c-f1b89acbc929 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:18.019851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6621dc18-0863-43ad-968c-f1b89acbc929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:18.081381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1122/2253] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:18.089815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd168d-5c98-4692-a261-425fd75a87e4 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:18.090106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd168d-5c98-4692-a261-425fd75a87e4 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:18.090383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd168d-5c98-4692-a261-425fd75a87e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.110840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5bbd168d-5c98-4692-a261-425fd75a87e4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:18.112197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5bbd168d-5c98-4692-a261-425fd75a87e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:18.120881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1132/2254] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:18.125327 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621dc18-0863-43ad-968c-f1b89acbc929 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:18.136505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621dc18-0863-43ad-968c-f1b89acbc929 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:18.141682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:18.142291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:18.142574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:18.142846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.143545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:18.143845 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:18.145444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:18.157606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1123/2255] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:03:18.162099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd168d-5c98-4692-a261-425fd75a87e4 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:18.171782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd168d-5c98-4692-a261-425fd75a87e4 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:18.176493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8d9c8-fa38-4385-b811-5345f3f8eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:18.177090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8d9c8-fa38-4385-b811-5345f3f8eff5 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:18.177367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8d9c8-fa38-4385-b811-5345f3f8eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:18.177635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8d9c8-fa38-4385-b811-5345f3f8eff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.178261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8d9c8-fa38-4385-b811-5345f3f8eff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:18.181144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8d9c8-fa38-4385-b811-5345f3f8eff5 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:18.306176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1133/2256] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:18 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:03:18.310376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:18.326048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c8e6-24b8-4946-8e0e-641e7a37e97f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:18.335819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:18.404036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97513d-458a-4e48-bc7e-d1fbcc1d2944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:18.404846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97513d-458a-4e48-bc7e-d1fbcc1d2944 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:18.405076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97513d-458a-4e48-bc7e-d1fbcc1d2944 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:18.405350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97513d-458a-4e48-bc7e-d1fbcc1d2944 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.405987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97513d-458a-4e48-bc7e-d1fbcc1d2944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:18.408519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97513d-458a-4e48-bc7e-d1fbcc1d2944 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:18.663454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1124/2257] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:18 2022] POST /identity/v3/users => generated 294 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:03:18.667742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8d9c8-fa38-4385-b811-5345f3f8eff5 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:18.679581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8d9c8-fa38-4385-b811-5345f3f8eff5 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:18.684824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f0ac-d909-4902-b4d8-43e6e455727d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:18.685465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f0ac-d909-4902-b4d8-43e6e455727d admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:18.685735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f0ac-d909-4902-b4d8-43e6e455727d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:18.686003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f0ac-d909-4902-b4d8-43e6e455727d admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.686666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117f0ac-d909-4902-b4d8-43e6e455727d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:18.688373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117f0ac-d909-4902-b4d8-43e6e455727d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:18.700760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1134/2258] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:18.708019 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97513d-458a-4e48-bc7e-d1fbcc1d2944 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:18.742240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97513d-458a-4e48-bc7e-d1fbcc1d2944 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:18.747954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c87c2-c8b2-4b29-ac52-02edb30d4d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:18.748693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c87c2-c8b2-4b29-ac52-02edb30d4d02 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:18.749066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c87c2-c8b2-4b29-ac52-02edb30d4d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:18.749460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c87c2-c8b2-4b29-ac52-02edb30d4d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.750220 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c87c2-c8b2-4b29-ac52-02edb30d4d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:18.752075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c87c2-c8b2-4b29-ac52-02edb30d4d02 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:18.766978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1125/2259] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:18.769664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f0ac-d909-4902-b4d8-43e6e455727d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:18.783189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f0ac-d909-4902-b4d8-43e6e455727d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:18.789121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675953a-eb9d-4880-ac79-d4fd6256a49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:18.790033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675953a-eb9d-4880-ac79-d4fd6256a49c admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:18.790658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675953a-eb9d-4880-ac79-d4fd6256a49c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:18.791037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675953a-eb9d-4880-ac79-d4fd6256a49c admin admin] PATH_INFO: `/v3/projects/fd3861a6e346434a9a8793414e2f868c/users/fc392a37fbde4e7e9e4f28f01c1d566c/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.813001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675953a-eb9d-4880-ac79-d4fd6256a49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3861a6e346434a9a8793414e2f868c, user_id=fc392a37fbde4e7e9e4f28f01c1d566c, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:18.813001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675953a-eb9d-4880-ac79-d4fd6256a49c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:18.907750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1135/2260] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:18 2022] PUT /identity/v3/projects/fd3861a6e346434a9a8793414e2f868c/users/fc392a37fbde4e7e9e4f28f01c1d566c/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:18.918398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dde51a-d56f-4a74-b5a3-19c47bd2a56b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:18.918687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dde51a-d56f-4a74-b5a3-19c47bd2a56b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:18.918992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dde51a-d56f-4a74-b5a3-19c47bd2a56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:18.968954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-79dde51a-d56f-4a74-b5a3-19c47bd2a56b None None] MFA Rules not processed for user `fc392a37fbde4e7e9e4f28f01c1d566c`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:18.970265 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-79dde51a-d56f-4a74-b5a3-19c47bd2a56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:19.060113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1126/2261] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:19.060718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675953a-eb9d-4880-ac79-d4fd6256a49c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:19.120826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675953a-eb9d-4880-ac79-d4fd6256a49c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:19.125524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aba1f1-856e-4049-a7de-ed111610575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:19.126126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aba1f1-856e-4049-a7de-ed111610575d admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:19.126399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aba1f1-856e-4049-a7de-ed111610575d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:19.126671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aba1f1-856e-4049-a7de-ed111610575d admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:19.127336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aba1f1-856e-4049-a7de-ed111610575d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:19.129296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aba1f1-856e-4049-a7de-ed111610575d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:19.245143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1136/2262] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:19 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:03:19.246445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dde51a-d56f-4a74-b5a3-19c47bd2a56b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:19.258228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dde51a-d56f-4a74-b5a3-19c47bd2a56b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:19.323898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e31b3-677e-41c8-bf59-76ad82871212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:19.324506 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e31b3-677e-41c8-bf59-76ad82871212 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:19.324788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e31b3-677e-41c8-bf59-76ad82871212 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:19.325053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e31b3-677e-41c8-bf59-76ad82871212 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:19.325669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e31b3-677e-41c8-bf59-76ad82871212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:19.327937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e31b3-677e-41c8-bf59-76ad82871212 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:19.439771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1127/2263] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:19 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:19.441033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aba1f1-856e-4049-a7de-ed111610575d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:19.456173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aba1f1-856e-4049-a7de-ed111610575d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:19.462652 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6882cc-32c2-43d9-a70f-cd9d636bd7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:19.463380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6882cc-32c2-43d9-a70f-cd9d636bd7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:19.463745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6882cc-32c2-43d9-a70f-cd9d636bd7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:19.463966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6882cc-32c2-43d9-a70f-cd9d636bd7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:19.464708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6882cc-32c2-43d9-a70f-cd9d636bd7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:19.467060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6882cc-32c2-43d9-a70f-cd9d636bd7c3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:19.480047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1137/2264] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:19.484302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e31b3-677e-41c8-bf59-76ad82871212 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:19.495983 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e31b3-677e-41c8-bf59-76ad82871212 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:19.501015 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90754ead-9414-4210-83c6-cb8a1c3c7ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:19.505760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754ead-9414-4210-83c6-cb8a1c3c7ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:19.505760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754ead-9414-4210-83c6-cb8a1c3c7ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:19.505760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90754ead-9414-4210-83c6-cb8a1c3c7ca6 admin admin] PATH_INFO: `/v3/projects/57876d90542042c78363474f46e4554c/users/1457e9e3c86f4425872ff31d9cfde5aa/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:19.507796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:19.521708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90754ead-9414-4210-83c6-cb8a1c3c7ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57876d90542042c78363474f46e4554c, user_id=1457e9e3c86f4425872ff31d9cfde5aa, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:19.525359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90754ead-9414-4210-83c6-cb8a1c3c7ca6 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:19.628853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1128/2265] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:19 2022] PUT /identity/v3/projects/57876d90542042c78363474f46e4554c/users/1457e9e3c86f4425872ff31d9cfde5aa/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:19.634005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6882cc-32c2-43d9-a70f-cd9d636bd7c3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:19.666116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6882cc-32c2-43d9-a70f-cd9d636bd7c3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:19.689150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:19.720045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d5377-970a-4a7f-9ba6-3138f41e9970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:19.723081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d5377-970a-4a7f-9ba6-3138f41e9970 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:19.723081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d5377-970a-4a7f-9ba6-3138f41e9970 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:19.723081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d5377-970a-4a7f-9ba6-3138f41e9970 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:19.723081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d5377-970a-4a7f-9ba6-3138f41e9970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:19.723540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d5377-970a-4a7f-9ba6-3138f41e9970 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:19.733576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1138/2266] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:19.738034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90754ead-9414-4210-83c6-cb8a1c3c7ca6 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:19.749401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90754ead-9414-4210-83c6-cb8a1c3c7ca6 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:19.754161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:19.754811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:19.755112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:19.755381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] PATH_INFO: `/v3/projects/57876d90542042c78363474f46e4554c/users/1457e9e3c86f4425872ff31d9cfde5aa/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:19.758029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57876d90542042c78363474f46e4554c, user_id=1457e9e3c86f4425872ff31d9cfde5aa, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:19.760692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:19.771165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1129/2267] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:19 2022] PUT /identity/v3/projects/57876d90542042c78363474f46e4554c/users/1457e9e3c86f4425872ff31d9cfde5aa/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:19.788546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5546-fb71-48d2-8aac-38a0e4507ef2 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:19.789039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5546-fb71-48d2-8aac-38a0e4507ef2 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:19.789350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5546-fb71-48d2-8aac-38a0e4507ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:19.820688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d38b5546-fb71-48d2-8aac-38a0e4507ef2 None None] MFA Rules not processed for user `1457e9e3c86f4425872ff31d9cfde5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:19.821948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d38b5546-fb71-48d2-8aac-38a0e4507ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:19.905382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1139/2268] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:19.917810 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:19.920593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:19.992511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ee7f-7931-4745-8c51-900dd0a3e245 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:19.996374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6706f99-790b-4e43-ba7f-59b3cec15c5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:19.997213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6706f99-790b-4e43-ba7f-59b3cec15c5d service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:19.997524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6706f99-790b-4e43-ba7f-59b3cec15c5d service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:19.997823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6706f99-790b-4e43-ba7f-59b3cec15c5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:20.008954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6706f99-790b-4e43-ba7f-59b3cec15c5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:20.010398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6706f99-790b-4e43-ba7f-59b3cec15c5d service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:20.021395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1130/2269] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:20.062050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5546-fb71-48d2-8aac-38a0e4507ef2 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:20.179388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5546-fb71-48d2-8aac-38a0e4507ef2 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:20.188788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:20.188788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:20.188788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:20.188788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] PATH_INFO: `/v3/users/fc392a37fbde4e7e9e4f28f01c1d566c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:20.189925 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc392a37fbde4e7e9e4f28f01c1d566c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:20.191892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:20.448141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc392a37fbde4e7e9e4f28f01c1d566c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:20.448833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc392a37fbde4e7e9e4f28f01c1d566c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:20.456715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:20.461278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc392a37fbde4e7e9e4f28f01c1d566c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:20.558733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc392a37fbde4e7e9e4f28f01c1d566c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:20.566135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc392a37fbde4e7e9e4f28f01c1d566c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:20.573080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1140/2270] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:20 2022] DELETE /identity/v3/users/fc392a37fbde4e7e9e4f28f01c1d566c => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:20.577613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14f1ba-11fc-4e1a-9b43-a6578854b599 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:20.578023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14f1ba-11fc-4e1a-9b43-a6578854b599 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:20.578152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14f1ba-11fc-4e1a-9b43-a6578854b599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:20.597123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5a14f1ba-11fc-4e1a-9b43-a6578854b599 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:20.598356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5a14f1ba-11fc-4e1a-9b43-a6578854b599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:20.657990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1131/2271] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:20.667957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:20.669397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:20.730660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b4019-1e00-478c-ad7f-45d8376b362c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:20.734397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f22269-f996-430b-b615-42c754f089b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:20.735041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22269-f996-430b-b615-42c754f089b1 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:20.735362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22269-f996-430b-b615-42c754f089b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:20.735631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22269-f996-430b-b615-42c754f089b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:20.745813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f22269-f996-430b-b615-42c754f089b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:20.747281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f22269-f996-430b-b615-42c754f089b1 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:20.757245 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1141/2272] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:21.209305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14f1ba-11fc-4e1a-9b43-a6578854b599 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:21.210946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5a14f1ba-11fc-4e1a-9b43-a6578854b599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:21.224213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14f1ba-11fc-4e1a-9b43-a6578854b599 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:21.231193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:21.232164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:21.232439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:21.232717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] PATH_INFO: `/v3/projects/fd3861a6e346434a9a8793414e2f868c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:21.233872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3861a6e346434a9a8793414e2f868c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:21.247271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:21.434458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3861a6e346434a9a8793414e2f868c', 'request_id': 'req-79299d58-d73b-425e-abe8-9c8c38f1987c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:21.440141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3861a6e346434a9a8793414e2f868c', 'request_id': 'req-79299d58-d73b-425e-abe8-9c8c38f1987c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:21.530508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3861a6e346434a9a8793414e2f868c', 'request_id': 'req-79299d58-d73b-425e-abe8-9c8c38f1987c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:21.538839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:21.543542 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] The token cache is being invalidate because project fd3861a6e346434a9a8793414e2f868c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:21.543825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:21.544463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:21.545952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1132/2273] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:21 2022] DELETE /identity/v3/projects/fd3861a6e346434a9a8793414e2f868c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:21.550566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f22269-f996-430b-b615-42c754f089b1 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:21.561939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f22269-f996-430b-b615-42c754f089b1 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:21.623600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:21.624227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:21.624504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:21.624771 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] PATH_INFO: `/v3/users/1457e9e3c86f4425872ff31d9cfde5aa` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:21.625881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1457e9e3c86f4425872ff31d9cfde5aa)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:21.627832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:21.890146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1457e9e3c86f4425872ff31d9cfde5aa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:21.890824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1457e9e3c86f4425872ff31d9cfde5aa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:21.907248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1457e9e3c86f4425872ff31d9cfde5aa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:22.125024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1457e9e3c86f4425872ff31d9cfde5aa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:22.136067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8d78f7ea-f140-4da7-9167-33e12e203d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1457e9e3c86f4425872ff31d9cfde5aa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:22.136067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:22.141148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1142/2274] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:21 2022] DELETE /identity/v3/users/1457e9e3c86f4425872ff31d9cfde5aa => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:03:22.737499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:22.739087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:22.757320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79299d58-d73b-425e-abe8-9c8c38f1987c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:22.817712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:22.818355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:22.819860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:22.820151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] PATH_INFO: `/v3/projects/57876d90542042c78363474f46e4554c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:22.821317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] RBAC: Authorizing `identity:delete_project(project_id=57876d90542042c78363474f46e4554c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:22.823290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:23.006254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57876d90542042c78363474f46e4554c', 'request_id': 'req-fc281671-bde6-411e-8706-2f462ac3e3af'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:23.012024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57876d90542042c78363474f46e4554c', 'request_id': 'req-fc281671-bde6-411e-8706-2f462ac3e3af'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:23.102308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57876d90542042c78363474f46e4554c', 'request_id': 'req-fc281671-bde6-411e-8706-2f462ac3e3af'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:23.114090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] The token cache is being invalidate because project 57876d90542042c78363474f46e4554c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:23.114379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:23.115126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc281671-bde6-411e-8706-2f462ac3e3af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:23.117319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1133/2275] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:22 2022] DELETE /identity/v3/projects/57876d90542042c78363474f46e4554c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:23.122500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071f0f-1fee-4da1-98e2-6f5e73ef78c4 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.123028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071f0f-1fee-4da1-98e2-6f5e73ef78c4 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.123028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071f0f-1fee-4da1-98e2-6f5e73ef78c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.142396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-29071f0f-1fee-4da1-98e2-6f5e73ef78c4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:23.143622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-29071f0f-1fee-4da1-98e2-6f5e73ef78c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:23.204255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1143/2276] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:23.212368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cac02-634a-48de-af29-d00aa551122d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.212647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cac02-634a-48de-af29-d00aa551122d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.212962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cac02-634a-48de-af29-d00aa551122d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.233366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-921cac02-634a-48de-af29-d00aa551122d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:23.234609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-921cac02-634a-48de-af29-d00aa551122d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:23.244073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1134/2277] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:23.248343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29071f0f-1fee-4da1-98e2-6f5e73ef78c4 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:23.258526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29071f0f-1fee-4da1-98e2-6f5e73ef78c4 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:23.263190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:23.266570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.266570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.266570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.266570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:23.266570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:23.267073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:23.278657 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1144/2278] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:03:23.283207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cac02-634a-48de-af29-d00aa551122d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:23.294140 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cac02-634a-48de-af29-d00aa551122d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:23.299017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a6496-db77-4b4d-8e99-5706ff2435e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:23.299615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6496-db77-4b4d-8e99-5706ff2435e9 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.299891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6496-db77-4b4d-8e99-5706ff2435e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.300161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a6496-db77-4b4d-8e99-5706ff2435e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.300841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a6496-db77-4b4d-8e99-5706ff2435e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:23.302736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a6496-db77-4b4d-8e99-5706ff2435e9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:23.419178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1135/2279] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:23 2022] POST /identity/v3/projects => generated 393 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:23.423595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:23.434331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946865b-3ec8-45a0-8010-be93f55d6962 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:23.501135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef77f87-2981-48cc-ba97-49b668e0c259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:23.501664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef77f87-2981-48cc-ba97-49b668e0c259 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.501943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef77f87-2981-48cc-ba97-49b668e0c259 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.502220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef77f87-2981-48cc-ba97-49b668e0c259 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.502834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef77f87-2981-48cc-ba97-49b668e0c259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:23.504829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef77f87-2981-48cc-ba97-49b668e0c259 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:23.603029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1145/2280] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:23 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:03:23.604318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a6496-db77-4b4d-8e99-5706ff2435e9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:23.614855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a6496-db77-4b4d-8e99-5706ff2435e9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:23.621444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fab52a-cdab-41bf-a2dd-1d1d83756bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:23.622051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fab52a-cdab-41bf-a2dd-1d1d83756bff admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.622325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fab52a-cdab-41bf-a2dd-1d1d83756bff admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.622592 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fab52a-cdab-41bf-a2dd-1d1d83756bff admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.623270 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fab52a-cdab-41bf-a2dd-1d1d83756bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:23.624866 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fab52a-cdab-41bf-a2dd-1d1d83756bff admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:23.635632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1136/2281] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:03:23.639869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef77f87-2981-48cc-ba97-49b668e0c259 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:23.651648 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef77f87-2981-48cc-ba97-49b668e0c259 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:23.658601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25f7b3-de12-49bc-a15b-d8db07f3100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:23.658601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25f7b3-de12-49bc-a15b-d8db07f3100e admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.658601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25f7b3-de12-49bc-a15b-d8db07f3100e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.659029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25f7b3-de12-49bc-a15b-d8db07f3100e admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.659713 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25f7b3-de12-49bc-a15b-d8db07f3100e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:23.661358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25f7b3-de12-49bc-a15b-d8db07f3100e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:23.681593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fab52a-cdab-41bf-a2dd-1d1d83756bff admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:23.681875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1146/2282] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:03:23.704476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fab52a-cdab-41bf-a2dd-1d1d83756bff admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:23.709543 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a164c-e3e7-4169-b39a-4426bf702ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:23.710201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a164c-e3e7-4169-b39a-4426bf702ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.710516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a164c-e3e7-4169-b39a-4426bf702ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.710808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a164c-e3e7-4169-b39a-4426bf702ec8 admin admin] PATH_INFO: `/v3/projects/00ac1b417e974a499ec9d75da204b499/users/da700b08064d4aca95520a13d6fcef7b/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.729056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a164c-e3e7-4169-b39a-4426bf702ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ac1b417e974a499ec9d75da204b499, user_id=da700b08064d4aca95520a13d6fcef7b, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:23.731798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a164c-e3e7-4169-b39a-4426bf702ec8 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:23.811820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1137/2283] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:23 2022] PUT /identity/v3/projects/00ac1b417e974a499ec9d75da204b499/users/da700b08064d4aca95520a13d6fcef7b/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:23.823469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95c4e-fd1b-4b2a-9ae2-c84b1484e27e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:23.824312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95c4e-fd1b-4b2a-9ae2-c84b1484e27e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:23.824312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95c4e-fd1b-4b2a-9ae2-c84b1484e27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:23.856162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-27a95c4e-fd1b-4b2a-9ae2-c84b1484e27e None None] MFA Rules not processed for user `da700b08064d4aca95520a13d6fcef7b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:23.857393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-27a95c4e-fd1b-4b2a-9ae2-c84b1484e27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:23.914544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:23.925755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1147/2284] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:03:23.931241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a164c-e3e7-4169-b39a-4426bf702ec8 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:23.961005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:23.996988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a164c-e3e7-4169-b39a-4426bf702ec8 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:24.001737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5efe4d-b194-43df-9281-292da0dc9183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:24.002340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5efe4d-b194-43df-9281-292da0dc9183 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.002618 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5efe4d-b194-43df-9281-292da0dc9183 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.002893 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5efe4d-b194-43df-9281-292da0dc9183 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.003569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5efe4d-b194-43df-9281-292da0dc9183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:24.005461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5efe4d-b194-43df-9281-292da0dc9183 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:24.170183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1138/2285] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:23 2022] POST /identity/v3/projects => generated 391 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:24.174433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a95c4e-fd1b-4b2a-9ae2-c84b1484e27e None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:24.185524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a95c4e-fd1b-4b2a-9ae2-c84b1484e27e None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:24.244005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793e513-fa4a-43a5-9098-1b5d09015674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:24.244624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e513-fa4a-43a5-9098-1b5d09015674 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.244907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e513-fa4a-43a5-9098-1b5d09015674 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.245179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793e513-fa4a-43a5-9098-1b5d09015674 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.245817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793e513-fa4a-43a5-9098-1b5d09015674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:24.248159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793e513-fa4a-43a5-9098-1b5d09015674 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:24.362617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1148/2286] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:24 2022] POST /identity/v3/users => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:24.366868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5efe4d-b194-43df-9281-292da0dc9183 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:24.377954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5efe4d-b194-43df-9281-292da0dc9183 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:24.383181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b7167-da66-44a3-98c2-8a891994ae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:24.383811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b7167-da66-44a3-98c2-8a891994ae4d admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.384089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b7167-da66-44a3-98c2-8a891994ae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.384360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b7167-da66-44a3-98c2-8a891994ae4d admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.385011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b7167-da66-44a3-98c2-8a891994ae4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:24.386743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b7167-da66-44a3-98c2-8a891994ae4d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:24.398381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1139/2287] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:24.407121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793e513-fa4a-43a5-9098-1b5d09015674 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:24.417926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793e513-fa4a-43a5-9098-1b5d09015674 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:24.422673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1834937-7b46-4246-88c4-76cf3392bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:24.423648 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1834937-7b46-4246-88c4-76cf3392bbba admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.424105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1834937-7b46-4246-88c4-76cf3392bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.425416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1834937-7b46-4246-88c4-76cf3392bbba admin admin] PATH_INFO: `/v3/projects/7324bf97771d45b6b6304ef3bfd7b1ae/users/61fe026ce5c74b0a8060bc1d5947a36c/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.440535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1834937-7b46-4246-88c4-76cf3392bbba admin admin] RBAC: Authorizing `identity:create_grant(project_id=7324bf97771d45b6b6304ef3bfd7b1ae, user_id=61fe026ce5c74b0a8060bc1d5947a36c, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:24.443203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1834937-7b46-4246-88c4-76cf3392bbba admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:24.542570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1149/2288] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:24 2022] PUT /identity/v3/projects/7324bf97771d45b6b6304ef3bfd7b1ae/users/61fe026ce5c74b0a8060bc1d5947a36c/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:24.546761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b7167-da66-44a3-98c2-8a891994ae4d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:24.557141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b7167-da66-44a3-98c2-8a891994ae4d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:24.618519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a4e99-a9fb-41dd-a18b-123b36fea96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:24.619156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a4e99-a9fb-41dd-a18b-123b36fea96c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.619435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a4e99-a9fb-41dd-a18b-123b36fea96c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.619701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a4e99-a9fb-41dd-a18b-123b36fea96c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.620339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a4e99-a9fb-41dd-a18b-123b36fea96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:24.621923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a4e99-a9fb-41dd-a18b-123b36fea96c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:24.631486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1140/2289] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:24.636427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1834937-7b46-4246-88c4-76cf3392bbba admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:24.646949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1834937-7b46-4246-88c4-76cf3392bbba admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:24.652379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:24.653033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.653307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.653574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] PATH_INFO: `/v3/projects/7324bf97771d45b6b6304ef3bfd7b1ae/users/61fe026ce5c74b0a8060bc1d5947a36c/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.655497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7324bf97771d45b6b6304ef3bfd7b1ae, user_id=61fe026ce5c74b0a8060bc1d5947a36c, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:24.658119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:24.669373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1150/2290] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:24 2022] PUT /identity/v3/projects/7324bf97771d45b6b6304ef3bfd7b1ae/users/61fe026ce5c74b0a8060bc1d5947a36c/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:24.680771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3151eda-d4ae-4836-a556-50c10d6bcb11 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.681053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3151eda-d4ae-4836-a556-50c10d6bcb11 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.681331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3151eda-d4ae-4836-a556-50c10d6bcb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.711799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e3151eda-d4ae-4836-a556-50c10d6bcb11 None None] MFA Rules not processed for user `61fe026ce5c74b0a8060bc1d5947a36c`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:24.713034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e3151eda-d4ae-4836-a556-50c10d6bcb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:24.796873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1141/2291] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:24 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:24.808956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:24.811304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:24.881080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11f48a-cfc0-4b97-8978-b23b110204c6 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:24.885919 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d319838-5de1-441c-89d1-81e8232bfe2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:24.886553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d319838-5de1-441c-89d1-81e8232bfe2b service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.886857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d319838-5de1-441c-89d1-81e8232bfe2b service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.887350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d319838-5de1-441c-89d1-81e8232bfe2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.898797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d319838-5de1-441c-89d1-81e8232bfe2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:24.899777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d319838-5de1-441c-89d1-81e8232bfe2b service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:24.918204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1151/2292] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:24 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:24.961716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3151eda-d4ae-4836-a556-50c10d6bcb11 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:24.963262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e3151eda-d4ae-4836-a556-50c10d6bcb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:24.975376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3151eda-d4ae-4836-a556-50c10d6bcb11 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:24.979360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76fa16-f1b5-4b4c-9571-6ccb8b36c886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:24.980024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76fa16-f1b5-4b4c-9571-6ccb8b36c886 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:24.991257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76fa16-f1b5-4b4c-9571-6ccb8b36c886 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:24.991257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76fa16-f1b5-4b4c-9571-6ccb8b36c886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:24.999398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:25.041742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76fa16-f1b5-4b4c-9571-6ccb8b36c886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:25.043683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76fa16-f1b5-4b4c-9571-6ccb8b36c886 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:25.055636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1142/2293] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:03:25.212931 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf162d89-a142-4095-bd69-cf8a27c0280e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:25.213473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf162d89-a142-4095-bd69-cf8a27c0280e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:25.213593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf162d89-a142-4095-bd69-cf8a27c0280e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:25.234545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cf162d89-a142-4095-bd69-cf8a27c0280e None None] MFA Rules not processed for user `b50041d36b0b430badd6442a2f09f063`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:25.235810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cf162d89-a142-4095-bd69-cf8a27c0280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:25.284333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:25.296152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1152/2294] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:30.675833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76fa16-f1b5-4b4c-9571-6ccb8b36c886 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:30.759132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76fa16-f1b5-4b4c-9571-6ccb8b36c886 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:30.763870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:30.764502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:30.764902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:30.765182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] PATH_INFO: `/v3/users/da700b08064d4aca95520a13d6fcef7b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:30.766362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da700b08064d4aca95520a13d6fcef7b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:30.768353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:31.050560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da700b08064d4aca95520a13d6fcef7b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:31.051271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da700b08064d4aca95520a13d6fcef7b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:31.066958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da700b08064d4aca95520a13d6fcef7b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:31.067760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da700b08064d4aca95520a13d6fcef7b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:31.195369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da700b08064d4aca95520a13d6fcef7b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:31.215848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1143/2295] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:30 2022] DELETE /identity/v3/users/da700b08064d4aca95520a13d6fcef7b => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:31.220224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba3dbc-4451-4b42-867e-7ecab7d94b01 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:31.220704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba3dbc-4451-4b42-867e-7ecab7d94b01 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:31.220921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba3dbc-4451-4b42-867e-7ecab7d94b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:31.241034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-73ba3dbc-4451-4b42-867e-7ecab7d94b01 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:31.242248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-73ba3dbc-4451-4b42-867e-7ecab7d94b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:31.304083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1153/2296] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:31.314120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:31.315867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:31.361158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:31.377908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36bceb-8b09-4e59-9fc9-2cc878b93f39 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:31.379646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189b037-cbba-4e40-b545-582162ebdfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:31.380258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b037-cbba-4e40-b545-582162ebdfbc service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:31.380526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b037-cbba-4e40-b545-582162ebdfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:31.380797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189b037-cbba-4e40-b545-582162ebdfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:31.390218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189b037-cbba-4e40-b545-582162ebdfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:31.391683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189b037-cbba-4e40-b545-582162ebdfbc service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:31.403620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1144/2297] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:31.895705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba3dbc-4451-4b42-867e-7ecab7d94b01 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:31.897443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-73ba3dbc-4451-4b42-867e-7ecab7d94b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:31.912051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba3dbc-4451-4b42-867e-7ecab7d94b01 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:31.917280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:31.918007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:31.918288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:31.918597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] PATH_INFO: `/v3/projects/00ac1b417e974a499ec9d75da204b499` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:31.919853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00ac1b417e974a499ec9d75da204b499)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:31.921822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:32.053623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00ac1b417e974a499ec9d75da204b499', 'request_id': 'req-98abded3-8e21-4e00-bc40-78e5aaadc843'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.060321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00ac1b417e974a499ec9d75da204b499', 'request_id': 'req-98abded3-8e21-4e00-bc40-78e5aaadc843'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.315227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00ac1b417e974a499ec9d75da204b499', 'request_id': 'req-98abded3-8e21-4e00-bc40-78e5aaadc843'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.327898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] The token cache is being invalidate because project 00ac1b417e974a499ec9d75da204b499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:32.328209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.328869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.330644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1154/2298] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:31 2022] DELETE /identity/v3/projects/00ac1b417e974a499ec9d75da204b499 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:32.334464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189b037-cbba-4e40-b545-582162ebdfbc service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:32.344483 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189b037-cbba-4e40-b545-582162ebdfbc service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:32.401670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:32.402296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:32.402573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:32.402843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] PATH_INFO: `/v3/users/61fe026ce5c74b0a8060bc1d5947a36c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:32.404015 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61fe026ce5c74b0a8060bc1d5947a36c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:32.405935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:32.672540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61fe026ce5c74b0a8060bc1d5947a36c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.673341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61fe026ce5c74b0a8060bc1d5947a36c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.687419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61fe026ce5c74b0a8060bc1d5947a36c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.688197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61fe026ce5c74b0a8060bc1d5947a36c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.807587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-89f2ab7c-0d1d-41af-be55-236c8d4b74b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61fe026ce5c74b0a8060bc1d5947a36c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:32.820935 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1145/2299] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:32 2022] DELETE /identity/v3/users/61fe026ce5c74b0a8060bc1d5947a36c => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:33.308894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:33.310335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:33.326403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abded3-8e21-4e00-bc40-78e5aaadc843 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:33.384056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:33.385294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:33.385568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:33.385922 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] PATH_INFO: `/v3/projects/7324bf97771d45b6b6304ef3bfd7b1ae` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:33.387093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7324bf97771d45b6b6304ef3bfd7b1ae)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:33.388997 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:33.410546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:33.519971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7324bf97771d45b6b6304ef3bfd7b1ae', 'request_id': 'req-90676e6f-1630-43c0-93f8-c4ed52de1272'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:33.525561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7324bf97771d45b6b6304ef3bfd7b1ae', 'request_id': 'req-90676e6f-1630-43c0-93f8-c4ed52de1272'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:33.641946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7324bf97771d45b6b6304ef3bfd7b1ae', 'request_id': 'req-90676e6f-1630-43c0-93f8-c4ed52de1272'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:33.654296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] The token cache is being invalidate because project 7324bf97771d45b6b6304ef3bfd7b1ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:33.654672 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:33.655228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-90676e6f-1630-43c0-93f8-c4ed52de1272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:33.656725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1155/2300] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:33 2022] DELETE /identity/v3/projects/7324bf97771d45b6b6304ef3bfd7b1ae => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:33.662165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3da4f-6e51-481f-be22-90995d8545d4 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:33.662655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3da4f-6e51-481f-be22-90995d8545d4 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:33.662774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3da4f-6e51-481f-be22-90995d8545d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:33.683392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7cf3da4f-6e51-481f-be22-90995d8545d4 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:33.684599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7cf3da4f-6e51-481f-be22-90995d8545d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:33.745179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:33.753904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1146/2301] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:33.762786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c3f1a-3f3b-49d4-986a-0f09c423937b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:33.763114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c3f1a-3f3b-49d4-986a-0f09c423937b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:33.763418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c3f1a-3f3b-49d4-986a-0f09c423937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:33.784215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-147c3f1a-3f3b-49d4-986a-0f09c423937b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:33.785444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-147c3f1a-3f3b-49d4-986a-0f09c423937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:33.794796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1156/2302] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:33.799362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3da4f-6e51-481f-be22-90995d8545d4 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:33.810438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3da4f-6e51-481f-be22-90995d8545d4 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:33.815485 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:33.816094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:33.816365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:33.816630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:33.817274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:33.817571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:33.819455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:33.832350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1147/2303] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:03:33.836794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c3f1a-3f3b-49d4-986a-0f09c423937b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:33.846582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c3f1a-3f3b-49d4-986a-0f09c423937b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:33.851765 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a6e22-9ba8-4264-b4b4-119db7fec197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:33.852369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6e22-9ba8-4264-b4b4-119db7fec197 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:33.852638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6e22-9ba8-4264-b4b4-119db7fec197 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:33.852925 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a6e22-9ba8-4264-b4b4-119db7fec197 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:33.854124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a6e22-9ba8-4264-b4b4-119db7fec197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:33.856087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a6e22-9ba8-4264-b4b4-119db7fec197 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:33.960523 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1157/2304] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:33 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:03:33.965228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:33.975666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bea7e-634f-4894-a00f-f272c2b1a5ed admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:34.046761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e93b31-818c-404f-b236-6b2f5327bf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:34.047427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e93b31-818c-404f-b236-6b2f5327bf31 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.047699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e93b31-818c-404f-b236-6b2f5327bf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:34.047966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e93b31-818c-404f-b236-6b2f5327bf31 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:34.048604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e93b31-818c-404f-b236-6b2f5327bf31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:34.050787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e93b31-818c-404f-b236-6b2f5327bf31 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:34.180003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1148/2305] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:33 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:34.181144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a6e22-9ba8-4264-b4b4-119db7fec197 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:34.192004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a6e22-9ba8-4264-b4b4-119db7fec197 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:34.197241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b917e-1c9b-4890-b210-d72f99f80858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:34.198260 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b917e-1c9b-4890-b210-d72f99f80858 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.198545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b917e-1c9b-4890-b210-d72f99f80858 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:34.198892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b917e-1c9b-4890-b210-d72f99f80858 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:34.199685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b917e-1c9b-4890-b210-d72f99f80858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:34.201534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b917e-1c9b-4890-b210-d72f99f80858 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:34.218721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1158/2306] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:34.221922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e93b31-818c-404f-b236-6b2f5327bf31 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:34.234125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e93b31-818c-404f-b236-6b2f5327bf31 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:34.239390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c6a25-8cdd-4fec-8ff3-f79fe96d497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:34.239996 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6a25-8cdd-4fec-8ff3-f79fe96d497a admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.240267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6a25-8cdd-4fec-8ff3-f79fe96d497a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:34.240759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c6a25-8cdd-4fec-8ff3-f79fe96d497a admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:34.241396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6a25-8cdd-4fec-8ff3-f79fe96d497a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:34.243060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c6a25-8cdd-4fec-8ff3-f79fe96d497a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:34.255481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1149/2307] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:34.260699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b917e-1c9b-4890-b210-d72f99f80858 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:34.271504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b917e-1c9b-4890-b210-d72f99f80858 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:34.277086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93a14e-b5a2-4c70-8bbc-c7f5eef7cd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:34.277744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93a14e-b5a2-4c70-8bbc-c7f5eef7cd35 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.278105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93a14e-b5a2-4c70-8bbc-c7f5eef7cd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:34.278379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93a14e-b5a2-4c70-8bbc-c7f5eef7cd35 admin admin] PATH_INFO: `/v3/projects/983aaf9b59a2413a92c905a912e02519/users/31a7011c61604217837497c38fe74fc2/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:34.294394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93a14e-b5a2-4c70-8bbc-c7f5eef7cd35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983aaf9b59a2413a92c905a912e02519, user_id=31a7011c61604217837497c38fe74fc2, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:34.297032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93a14e-b5a2-4c70-8bbc-c7f5eef7cd35 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:34.433995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1159/2308] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:34 2022] PUT /identity/v3/projects/983aaf9b59a2413a92c905a912e02519/users/31a7011c61604217837497c38fe74fc2/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:03:34.445662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cb990-2726-4220-9205-b47c3c227f6d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.445971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cb990-2726-4220-9205-b47c3c227f6d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:34.446243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cb990-2726-4220-9205-b47c3c227f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:34.476970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-023cb990-2726-4220-9205-b47c3c227f6d None None] MFA Rules not processed for user `31a7011c61604217837497c38fe74fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:34.477409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-023cb990-2726-4220-9205-b47c3c227f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:34.543909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1150/2309] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:34.548558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93a14e-b5a2-4c70-8bbc-c7f5eef7cd35 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:34.614134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93a14e-b5a2-4c70-8bbc-c7f5eef7cd35 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:34.620049 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e649a-5b44-454d-9c54-264a08a919a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:34.620854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e649a-5b44-454d-9c54-264a08a919a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.620984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e649a-5b44-454d-9c54-264a08a919a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:34.621248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e649a-5b44-454d-9c54-264a08a919a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:34.621874 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e649a-5b44-454d-9c54-264a08a919a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:34.624114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e649a-5b44-454d-9c54-264a08a919a9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:34.762858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1160/2310] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:34 2022] POST /identity/v3/projects => generated 381 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:34.767576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cb990-2726-4220-9205-b47c3c227f6d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:34.779392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cb990-2726-4220-9205-b47c3c227f6d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:34.847676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545ab92-ae84-4b97-b725-a3274e3c8894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:34.848330 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ab92-ae84-4b97-b725-a3274e3c8894 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.848606 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ab92-ae84-4b97-b725-a3274e3c8894 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:34.848981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545ab92-ae84-4b97-b725-a3274e3c8894 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:34.849732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545ab92-ae84-4b97-b725-a3274e3c8894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:34.851827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545ab92-ae84-4b97-b725-a3274e3c8894 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:34.944069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1151/2311] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:34 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:34.948039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e649a-5b44-454d-9c54-264a08a919a9 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:34.958545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e649a-5b44-454d-9c54-264a08a919a9 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:34.963763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3ea73-b787-4569-9949-41ff6359abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:34.964363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3ea73-b787-4569-9949-41ff6359abd9 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.964637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3ea73-b787-4569-9949-41ff6359abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:34.964897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3ea73-b787-4569-9949-41ff6359abd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:34.965539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3ea73-b787-4569-9949-41ff6359abd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:34.967133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3ea73-b787-4569-9949-41ff6359abd9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:34.979240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1161/2312] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:03:34.983464 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545ab92-ae84-4b97-b725-a3274e3c8894 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:34.994241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545ab92-ae84-4b97-b725-a3274e3c8894 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:34.999062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95645a-6727-45a1-96e3-58d553c95cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:34.999722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95645a-6727-45a1-96e3-58d553c95cff admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:34.999999 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95645a-6727-45a1-96e3-58d553c95cff admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:35.000301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95645a-6727-45a1-96e3-58d553c95cff admin admin] PATH_INFO: `/v3/projects/e497531d594c4d9ba12f688c279a6ca4/users/961bcf72f24944d89b4aa255707e62fa/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:35.020787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95645a-6727-45a1-96e3-58d553c95cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e497531d594c4d9ba12f688c279a6ca4, user_id=961bcf72f24944d89b4aa255707e62fa, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:35.023588 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95645a-6727-45a1-96e3-58d553c95cff admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:35.079357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:35.113579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1152/2313] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:34 2022] PUT /identity/v3/projects/e497531d594c4d9ba12f688c279a6ca4/users/961bcf72f24944d89b4aa255707e62fa/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:35.117732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3ea73-b787-4569-9949-41ff6359abd9 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:35.128277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3ea73-b787-4569-9949-41ff6359abd9 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:35.141954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:35.193108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934afb3-af20-4df5-949e-add1ae4a7fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:35.194125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934afb3-af20-4df5-949e-add1ae4a7fba admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:35.194457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934afb3-af20-4df5-949e-add1ae4a7fba admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:35.194732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934afb3-af20-4df5-949e-add1ae4a7fba admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:35.195414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934afb3-af20-4df5-949e-add1ae4a7fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:35.197085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934afb3-af20-4df5-949e-add1ae4a7fba admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:35.207900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1162/2314] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:03:35.213107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95645a-6727-45a1-96e3-58d553c95cff admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:35.224077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95645a-6727-45a1-96e3-58d553c95cff admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:35.229707 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:35.230640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:35.231138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:35.231943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] PATH_INFO: `/v3/projects/e497531d594c4d9ba12f688c279a6ca4/users/961bcf72f24944d89b4aa255707e62fa/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:35.233879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e497531d594c4d9ba12f688c279a6ca4, user_id=961bcf72f24944d89b4aa255707e62fa, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:35.236792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:35.250130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1153/2315] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:35 2022] PUT /identity/v3/projects/e497531d594c4d9ba12f688c279a6ca4/users/961bcf72f24944d89b4aa255707e62fa/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:35.261124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c458f5-5872-4ce1-9bcb-e7ceb84bacbe None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:35.261516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c458f5-5872-4ce1-9bcb-e7ceb84bacbe None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:35.261900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c458f5-5872-4ce1-9bcb-e7ceb84bacbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:35.294366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-28c458f5-5872-4ce1-9bcb-e7ceb84bacbe None None] MFA Rules not processed for user `961bcf72f24944d89b4aa255707e62fa`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:35.295795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-28c458f5-5872-4ce1-9bcb-e7ceb84bacbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:35.364116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1163/2316] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:35.375285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:35.377085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:35.437193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1db6-0241-4b83-8d39-b040b56f6cf3 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:35.441237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5e334-acda-41e2-9ab8-b6d0aff9bd37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:35.441841 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5e334-acda-41e2-9ab8-b6d0aff9bd37 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:35.442114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5e334-acda-41e2-9ab8-b6d0aff9bd37 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:35.442379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5e334-acda-41e2-9ab8-b6d0aff9bd37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:35.453091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5e334-acda-41e2-9ab8-b6d0aff9bd37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:35.454721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5e334-acda-41e2-9ab8-b6d0aff9bd37 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:35.465898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1154/2317] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:35.995341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c458f5-5872-4ce1-9bcb-e7ceb84bacbe None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:36.060139 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c458f5-5872-4ce1-9bcb-e7ceb84bacbe None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:36.064914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:36.065599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:36.065853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:36.066121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] PATH_INFO: `/v3/users/31a7011c61604217837497c38fe74fc2` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:36.067273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a7011c61604217837497c38fe74fc2)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:36.069529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:36.319842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:36.325000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a7011c61604217837497c38fe74fc2'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.325661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a7011c61604217837497c38fe74fc2'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.340866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a7011c61604217837497c38fe74fc2'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.453236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a7011c61604217837497c38fe74fc2'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.461393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a7011c61604217837497c38fe74fc2'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:36.469169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1164/2318] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:35 2022] DELETE /identity/v3/users/31a7011c61604217837497c38fe74fc2 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:36.474069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379429d-98c0-4f8b-8bf2-0864f626a5d3 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:36.474556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379429d-98c0-4f8b-8bf2-0864f626a5d3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:36.474695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379429d-98c0-4f8b-8bf2-0864f626a5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:36.493331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f379429d-98c0-4f8b-8bf2-0864f626a5d3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:36.495156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f379429d-98c0-4f8b-8bf2-0864f626a5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:36.564344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1155/2319] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:36.576528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:36.578056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:36.649198 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba6874-b9e6-41a0-b714-dc7764e3da3c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:36.653889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19ab05-1c96-4d59-9be3-c1f15e7b33df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:36.654540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19ab05-1c96-4d59-9be3-c1f15e7b33df service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:36.654877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19ab05-1c96-4d59-9be3-c1f15e7b33df service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:36.655256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19ab05-1c96-4d59-9be3-c1f15e7b33df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:36.665350 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19ab05-1c96-4d59-9be3-c1f15e7b33df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:36.666826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19ab05-1c96-4d59-9be3-c1f15e7b33df service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:36.677000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1165/2320] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:37.201274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379429d-98c0-4f8b-8bf2-0864f626a5d3 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:37.203806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f379429d-98c0-4f8b-8bf2-0864f626a5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:37.217796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379429d-98c0-4f8b-8bf2-0864f626a5d3 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:37.224369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:37.226267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:37.226267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:37.226267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] PATH_INFO: `/v3/projects/983aaf9b59a2413a92c905a912e02519` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:37.227298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=983aaf9b59a2413a92c905a912e02519)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:37.229722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:37.351522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:37.355403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983aaf9b59a2413a92c905a912e02519', 'request_id': 'req-741a426e-e420-4695-be7b-e943f0397faf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.360872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983aaf9b59a2413a92c905a912e02519', 'request_id': 'req-741a426e-e420-4695-be7b-e943f0397faf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.476286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983aaf9b59a2413a92c905a912e02519', 'request_id': 'req-741a426e-e420-4695-be7b-e943f0397faf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.488718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] The token cache is being invalidate because project 983aaf9b59a2413a92c905a912e02519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:37.489007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.489651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.491844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1156/2321] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:37 2022] DELETE /identity/v3/projects/983aaf9b59a2413a92c905a912e02519 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:37.496925 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19ab05-1c96-4d59-9be3-c1f15e7b33df service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:37.507099 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19ab05-1c96-4d59-9be3-c1f15e7b33df service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:37.568456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:37.569321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:37.569651 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:37.569918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] PATH_INFO: `/v3/users/961bcf72f24944d89b4aa255707e62fa` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:37.575749 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961bcf72f24944d89b4aa255707e62fa)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:37.581194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:37.748857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961bcf72f24944d89b4aa255707e62fa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.749493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961bcf72f24944d89b4aa255707e62fa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.761028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:37.762191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961bcf72f24944d89b4aa255707e62fa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.845152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961bcf72f24944d89b4aa255707e62fa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.853476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-939037c1-8616-4765-b4e7-856964514023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961bcf72f24944d89b4aa255707e62fa'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:37.863003 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1166/2322] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:37 2022] DELETE /identity/v3/users/961bcf72f24944d89b4aa255707e62fa => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:03:38.306208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:38.307830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:38.321751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a426e-e420-4695-be7b-e943f0397faf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:38.385320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:38.386041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.386318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.386591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] PATH_INFO: `/v3/projects/e497531d594c4d9ba12f688c279a6ca4` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.387765 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e497531d594c4d9ba12f688c279a6ca4)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:38.389678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:38.534505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e497531d594c4d9ba12f688c279a6ca4', 'request_id': 'req-3968ec77-764b-47cf-9aaa-81996edb9e67'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:38.541767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e497531d594c4d9ba12f688c279a6ca4', 'request_id': 'req-3968ec77-764b-47cf-9aaa-81996edb9e67'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:38.655786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e497531d594c4d9ba12f688c279a6ca4', 'request_id': 'req-3968ec77-764b-47cf-9aaa-81996edb9e67'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:38.667744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] The token cache is being invalidate because project e497531d594c4d9ba12f688c279a6ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:38.668033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:38.668687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3968ec77-764b-47cf-9aaa-81996edb9e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:38.670199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1157/2323] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:38 2022] DELETE /identity/v3/projects/e497531d594c4d9ba12f688c279a6ca4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:38.675121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da47e8-ec7a-4173-a416-e3b4307e8c3b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.675602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da47e8-ec7a-4173-a416-e3b4307e8c3b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.675699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da47e8-ec7a-4173-a416-e3b4307e8c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.696464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-c2da47e8-ec7a-4173-a416-e3b4307e8c3b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:38.698426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c2da47e8-ec7a-4173-a416-e3b4307e8c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:38.758544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1167/2324] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:38.766987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f3e97-fe78-438f-8c30-4116bfd6c3b6 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.767375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f3e97-fe78-438f-8c30-4116bfd6c3b6 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.767674 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f3e97-fe78-438f-8c30-4116bfd6c3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.786959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-518f3e97-fe78-438f-8c30-4116bfd6c3b6 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:38.788175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-518f3e97-fe78-438f-8c30-4116bfd6c3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:38.796237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1158/2325] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:38.801018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da47e8-ec7a-4173-a416-e3b4307e8c3b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:38.812310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da47e8-ec7a-4173-a416-e3b4307e8c3b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:38.817805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:38.818944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.819359 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.819953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.820326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:38.820629 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:38.823448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:38.836567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1168/2326] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:03:38.841192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f3e97-fe78-438f-8c30-4116bfd6c3b6 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:38.850472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f3e97-fe78-438f-8c30-4116bfd6c3b6 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:38.855434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1253d-113b-4cab-a907-1390e79f4e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:38.856023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1253d-113b-4cab-a907-1390e79f4e82 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:38.856293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1253d-113b-4cab-a907-1390e79f4e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:38.856558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1253d-113b-4cab-a907-1390e79f4e82 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:38.857169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1253d-113b-4cab-a907-1390e79f4e82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:38.859092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1253d-113b-4cab-a907-1390e79f4e82 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:39.008727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1159/2327] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:38 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:39.008852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:39.019185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4fa00-367a-4e52-b51d-36e504b148c2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:39.079170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e87a94-1ba7-4fee-b74d-ff5447c7aa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:39.079796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87a94-1ba7-4fee-b74d-ff5447c7aa4f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:39.080072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87a94-1ba7-4fee-b74d-ff5447c7aa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:39.080343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87a94-1ba7-4fee-b74d-ff5447c7aa4f admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:39.080958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e87a94-1ba7-4fee-b74d-ff5447c7aa4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:39.082870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e87a94-1ba7-4fee-b74d-ff5447c7aa4f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:39.233912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1169/2328] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:39 2022] POST /identity/v3/users => generated 291 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:03:39.238013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1253d-113b-4cab-a907-1390e79f4e82 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:39.249619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1253d-113b-4cab-a907-1390e79f4e82 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:39.254325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae003bc5-4106-4828-bc64-fb868884515c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:39.254948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae003bc5-4106-4828-bc64-fb868884515c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:39.255237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae003bc5-4106-4828-bc64-fb868884515c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:39.255619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae003bc5-4106-4828-bc64-fb868884515c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:39.256262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae003bc5-4106-4828-bc64-fb868884515c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:39.257838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae003bc5-4106-4828-bc64-fb868884515c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:39.268325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1160/2329] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:39.272085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e87a94-1ba7-4fee-b74d-ff5447c7aa4f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:39.284641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e87a94-1ba7-4fee-b74d-ff5447c7aa4f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:39.289872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5067c-0607-4184-bceb-8366db878dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:39.290585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5067c-0607-4184-bceb-8366db878dfb admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:39.291668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5067c-0607-4184-bceb-8366db878dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:39.291998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5067c-0607-4184-bceb-8366db878dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:39.292802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5067c-0607-4184-bceb-8366db878dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:39.294544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5067c-0607-4184-bceb-8366db878dfb admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:39.308725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1170/2330] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:39.313287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae003bc5-4106-4828-bc64-fb868884515c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:39.326313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae003bc5-4106-4828-bc64-fb868884515c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:39.331078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43e46-cd65-4568-84dd-96b0b35059ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:39.331773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43e46-cd65-4568-84dd-96b0b35059ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:39.332055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43e46-cd65-4568-84dd-96b0b35059ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:39.332339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43e46-cd65-4568-84dd-96b0b35059ec admin admin] PATH_INFO: `/v3/projects/dc2060e442ed41d4b0b512cbafb07c4c/users/7683153753df41aeb948b3319cfeed93/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:39.348163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf43e46-cd65-4568-84dd-96b0b35059ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2060e442ed41d4b0b512cbafb07c4c, user_id=7683153753df41aeb948b3319cfeed93, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:39.350944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf43e46-cd65-4568-84dd-96b0b35059ec admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:39.435171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:39.440696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1161/2331] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:39 2022] PUT /identity/v3/projects/dc2060e442ed41d4b0b512cbafb07c4c/users/7683153753df41aeb948b3319cfeed93/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:39.452908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479d77a-1682-47ac-beb5-8e7db382353b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:39.453187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479d77a-1682-47ac-beb5-8e7db382353b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:39.453450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479d77a-1682-47ac-beb5-8e7db382353b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:39.483430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d479d77a-1682-47ac-beb5-8e7db382353b None None] MFA Rules not processed for user `7683153753df41aeb948b3319cfeed93`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:39.484687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d479d77a-1682-47ac-beb5-8e7db382353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:39.519804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:39.553938 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1171/2332] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:39.558910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43e46-cd65-4568-84dd-96b0b35059ec admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:39.626952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43e46-cd65-4568-84dd-96b0b35059ec admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:39.631830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c01f72-81b2-4f9d-896c-89596e3c3fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:39.634564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c01f72-81b2-4f9d-896c-89596e3c3fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:39.634564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c01f72-81b2-4f9d-896c-89596e3c3fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:39.634564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c01f72-81b2-4f9d-896c-89596e3c3fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:39.634564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c01f72-81b2-4f9d-896c-89596e3c3fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:39.637441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c01f72-81b2-4f9d-896c-89596e3c3fc5 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:39.812832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1162/2333] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:39 2022] POST /identity/v3/projects => generated 383 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:39.817031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479d77a-1682-47ac-beb5-8e7db382353b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:39.828575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479d77a-1682-47ac-beb5-8e7db382353b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:39.887138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc990f92-ee65-493b-a86a-8638fa802a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:39.887785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc990f92-ee65-493b-a86a-8638fa802a98 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:39.888066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc990f92-ee65-493b-a86a-8638fa802a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:39.888336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc990f92-ee65-493b-a86a-8638fa802a98 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:39.889269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc990f92-ee65-493b-a86a-8638fa802a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:39.891267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc990f92-ee65-493b-a86a-8638fa802a98 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:40.023066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1172/2334] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:39 2022] POST /identity/v3/users => generated 298 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:40.026159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c01f72-81b2-4f9d-896c-89596e3c3fc5 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:40.037967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c01f72-81b2-4f9d-896c-89596e3c3fc5 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:40.044536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b621cc0-a3c4-4d8b-a248-847012ae8c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:40.046168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b621cc0-a3c4-4d8b-a248-847012ae8c89 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:40.046311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b621cc0-a3c4-4d8b-a248-847012ae8c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:40.046854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b621cc0-a3c4-4d8b-a248-847012ae8c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:40.047820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b621cc0-a3c4-4d8b-a248-847012ae8c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:40.050367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b621cc0-a3c4-4d8b-a248-847012ae8c89 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:40.066351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1163/2335] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:40.070472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc990f92-ee65-493b-a86a-8638fa802a98 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:40.081715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc990f92-ee65-493b-a86a-8638fa802a98 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:40.087574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0319f-db67-4ba7-b92e-38c772ffa8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:40.088027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0319f-db67-4ba7-b92e-38c772ffa8da admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:40.088325 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0319f-db67-4ba7-b92e-38c772ffa8da admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:40.088666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0319f-db67-4ba7-b92e-38c772ffa8da admin admin] PATH_INFO: `/v3/projects/124fe5b8ee0d4d85981ea23cd225934d/users/e00125fa92514a39827964c0ec3576c5/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:40.108788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0319f-db67-4ba7-b92e-38c772ffa8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=124fe5b8ee0d4d85981ea23cd225934d, user_id=e00125fa92514a39827964c0ec3576c5, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:40.113161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0319f-db67-4ba7-b92e-38c772ffa8da admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:40.285138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1173/2336] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:40 2022] PUT /identity/v3/projects/124fe5b8ee0d4d85981ea23cd225934d/users/e00125fa92514a39827964c0ec3576c5/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:40.289305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b621cc0-a3c4-4d8b-a248-847012ae8c89 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:40.301138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b621cc0-a3c4-4d8b-a248-847012ae8c89 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:40.365722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ca44c-b914-4e33-81b8-d8cc96a8e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:40.366669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ca44c-b914-4e33-81b8-d8cc96a8e259 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:40.367020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ca44c-b914-4e33-81b8-d8cc96a8e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:40.367523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ca44c-b914-4e33-81b8-d8cc96a8e259 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:40.368166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ca44c-b914-4e33-81b8-d8cc96a8e259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:40.370474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ca44c-b914-4e33-81b8-d8cc96a8e259 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:40.391789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1164/2337] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:40.398635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0319f-db67-4ba7-b92e-38c772ffa8da admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:40.412369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0319f-db67-4ba7-b92e-38c772ffa8da admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:40.417477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:40.418464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:40.419134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:40.419589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] PATH_INFO: `/v3/projects/124fe5b8ee0d4d85981ea23cd225934d/users/e00125fa92514a39827964c0ec3576c5/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:40.421779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=124fe5b8ee0d4d85981ea23cd225934d, user_id=e00125fa92514a39827964c0ec3576c5, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:40.424946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:40.439824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1174/2338] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:40 2022] PUT /identity/v3/projects/124fe5b8ee0d4d85981ea23cd225934d/users/e00125fa92514a39827964c0ec3576c5/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:40.451897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58855896-829d-4377-a131-d5787b3b8c17 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:40.452209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58855896-829d-4377-a131-d5787b3b8c17 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:40.452456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58855896-829d-4377-a131-d5787b3b8c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:40.482981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-58855896-829d-4377-a131-d5787b3b8c17 None None] MFA Rules not processed for user `e00125fa92514a39827964c0ec3576c5`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:40.484279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-58855896-829d-4377-a131-d5787b3b8c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:40.554136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:40.566967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1165/2339] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:40.579635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:40.581349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:40.653505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00755a00-dd78-459c-a17a-814b18aa5911 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:40.658876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd4e8-5f1d-4a3c-afac-e04d52abbd59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:40.659936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd4e8-5f1d-4a3c-afac-e04d52abbd59 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:40.660191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd4e8-5f1d-4a3c-afac-e04d52abbd59 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:40.660610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd4e8-5f1d-4a3c-afac-e04d52abbd59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:40.675377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fd4e8-5f1d-4a3c-afac-e04d52abbd59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:40.677911 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fd4e8-5f1d-4a3c-afac-e04d52abbd59 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:40.691498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1175/2340] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:40.912993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58855896-829d-4377-a131-d5787b3b8c17 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:40.924747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58855896-829d-4377-a131-d5787b3b8c17 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:40.930638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eb9d4-3ff8-41a2-a7f4-21157fcadac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:40.932987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb9d4-3ff8-41a2-a7f4-21157fcadac3 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:40.932987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb9d4-3ff8-41a2-a7f4-21157fcadac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:40.933205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb9d4-3ff8-41a2-a7f4-21157fcadac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:41.011600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eb9d4-3ff8-41a2-a7f4-21157fcadac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:41.013309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eb9d4-3ff8-41a2-a7f4-21157fcadac3 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:41.024273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1166/2341] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:03:41.578951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd4e8-5f1d-4a3c-afac-e04d52abbd59 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:41.616827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:41.648770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd4e8-5f1d-4a3c-afac-e04d52abbd59 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:41.653935 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:41.654629 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:41.654939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:41.655248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] PATH_INFO: `/v3/users/7683153753df41aeb948b3319cfeed93` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:41.656364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=7683153753df41aeb948b3319cfeed93)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:41.658303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:41.861829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7683153753df41aeb948b3319cfeed93'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:41.862821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7683153753df41aeb948b3319cfeed93'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:41.894396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7683153753df41aeb948b3319cfeed93'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:41.980236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7683153753df41aeb948b3319cfeed93'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:41.990184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-605cd538-d097-436d-9a60-39100b75a4ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7683153753df41aeb948b3319cfeed93'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:41.999317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1176/2342] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:41 2022] DELETE /identity/v3/users/7683153753df41aeb948b3319cfeed93 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:42.003338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5e8b3-6859-4b06-81df-4cbe11f87e95 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:42.004522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5e8b3-6859-4b06-81df-4cbe11f87e95 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:42.004871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5e8b3-6859-4b06-81df-4cbe11f87e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:42.018750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b4a6-e25d-4764-82f7-2c3b0db58703 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:42.019063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b4a6-e25d-4764-82f7-2c3b0db58703 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:42.019336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b4a6-e25d-4764-82f7-2c3b0db58703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:42.035378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d5e5e8b3-6859-4b06-81df-4cbe11f87e95 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:42.037082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d5e5e8b3-6859-4b06-81df-4cbe11f87e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:42.059037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-c007b4a6-e25d-4764-82f7-2c3b0db58703 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:42.060932 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c007b4a6-e25d-4764-82f7-2c3b0db58703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:42.115609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1167/2343] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:03:42.125494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5e8b3-6859-4b06-81df-4cbe11f87e95 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:42.126976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d5e5e8b3-6859-4b06-81df-4cbe11f87e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:42.148622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1177/2344] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:42.194394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b4a6-e25d-4764-82f7-2c3b0db58703 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:42.196576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c007b4a6-e25d-4764-82f7-2c3b0db58703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:42.201159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5e8b3-6859-4b06-81df-4cbe11f87e95 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:42.205158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:42.205800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:42.206076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:42.206344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:42.219310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:42.220794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:42.236686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1168/2345] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:03:42.294622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:42.312860 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b4a6-e25d-4764-82f7-2c3b0db58703 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:42.318865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae8252-251a-43c4-a0aa-a8a700cd4ab0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:42.319873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae8252-251a-43c4-a0aa-a8a700cd4ab0 service swift] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:42.328767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae8252-251a-43c4-a0aa-a8a700cd4ab0 service swift] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:42.328767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae8252-251a-43c4-a0aa-a8a700cd4ab0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:42.332399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae8252-251a-43c4-a0aa-a8a700cd4ab0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:42.336284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae8252-251a-43c4-a0aa-a8a700cd4ab0 service swift] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:42.348141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1178/2346] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:03:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:03:43.344625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:43.347007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:43.369293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f39d33-baf7-47f9-bdb3-ed17a166223c service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:43.375303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:43.375939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:43.376215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:43.377953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] PATH_INFO: `/v3/projects/dc2060e442ed41d4b0b512cbafb07c4c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:43.379377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2060e442ed41d4b0b512cbafb07c4c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:43.380507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:43.696397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:43.746607 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2060e442ed41d4b0b512cbafb07c4c', 'request_id': 'req-eeec14b0-afca-4d52-8554-6336c4a56866'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:43.761318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2060e442ed41d4b0b512cbafb07c4c', 'request_id': 'req-eeec14b0-afca-4d52-8554-6336c4a56866'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:43.885281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2060e442ed41d4b0b512cbafb07c4c', 'request_id': 'req-eeec14b0-afca-4d52-8554-6336c4a56866'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:43.918810 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] The token cache is being invalidate because project dc2060e442ed41d4b0b512cbafb07c4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:43.919158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:43.919851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:43.924562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1169/2347] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:43 2022] DELETE /identity/v3/projects/dc2060e442ed41d4b0b512cbafb07c4c => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:43.925505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae8252-251a-43c4-a0aa-a8a700cd4ab0 service swift] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:43.953586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae8252-251a-43c4-a0aa-a8a700cd4ab0 service swift] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:44.037955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:44.038597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:44.038869 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:44.039168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] PATH_INFO: `/v3/users/e00125fa92514a39827964c0ec3576c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:44.041258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00125fa92514a39827964c0ec3576c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:44.044085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:44.326329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00125fa92514a39827964c0ec3576c5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:44.327243 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00125fa92514a39827964c0ec3576c5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:44.342306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00125fa92514a39827964c0ec3576c5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:44.448212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00125fa92514a39827964c0ec3576c5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:44.457764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-6e468ada-102d-499b-8792-aa534079e295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00125fa92514a39827964c0ec3576c5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:44.465383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1179/2348] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:43 2022] DELETE /identity/v3/users/e00125fa92514a39827964c0ec3576c5 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:45.538045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:45.539562 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:45.555792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec14b0-afca-4d52-8554-6336c4a56866 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:45.615501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:45.616129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:45.616400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:45.616670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] PATH_INFO: `/v3/projects/124fe5b8ee0d4d85981ea23cd225934d` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:45.617771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=124fe5b8ee0d4d85981ea23cd225934d)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:45.619877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:45.756637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124fe5b8ee0d4d85981ea23cd225934d', 'request_id': 'req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:45.764692 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124fe5b8ee0d4d85981ea23cd225934d', 'request_id': 'req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:45.867254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124fe5b8ee0d4d85981ea23cd225934d', 'request_id': 'req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:45.881296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] The token cache is being invalidate because project 124fe5b8ee0d4d85981ea23cd225934d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:45.881671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:45.882395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7d58cd6f-8767-48d6-a812-75b8c8e3a3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:45.884267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1170/2349] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:45 2022] DELETE /identity/v3/projects/124fe5b8ee0d4d85981ea23cd225934d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:45.889995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea5bb2-dd3a-4aab-92e2-6ad69cb83e83 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:45.891112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea5bb2-dd3a-4aab-92e2-6ad69cb83e83 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:45.891626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea5bb2-dd3a-4aab-92e2-6ad69cb83e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:45.915863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-36ea5bb2-dd3a-4aab-92e2-6ad69cb83e83 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:45.917649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-36ea5bb2-dd3a-4aab-92e2-6ad69cb83e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:45.993165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1180/2350] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:46.002687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8eafa-2aa8-4e1c-aaa7-6a4d3a5ab66b None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:46.002998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8eafa-2aa8-4e1c-aaa7-6a4d3a5ab66b None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:46.003273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8eafa-2aa8-4e1c-aaa7-6a4d3a5ab66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:46.030259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-44c8eafa-2aa8-4e1c-aaa7-6a4d3a5ab66b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:46.031782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-44c8eafa-2aa8-4e1c-aaa7-6a4d3a5ab66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:46.042281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1171/2351] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:03:46.048166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea5bb2-dd3a-4aab-92e2-6ad69cb83e83 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:46.055610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:46.060984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea5bb2-dd3a-4aab-92e2-6ad69cb83e83 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:46.066735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:46.067524 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:46.068185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:46.068615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:46.069405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:46.069805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:46.072447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:46.091246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1181/2352] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 16:03:46.093998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8eafa-2aa8-4e1c-aaa7-6a4d3a5ab66b None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:46.103163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8eafa-2aa8-4e1c-aaa7-6a4d3a5ab66b None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:46.110342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c6bce-20fd-4e6e-902d-a6fd6f8b5466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:46.110342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c6bce-20fd-4e6e-902d-a6fd6f8b5466 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:46.110342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c6bce-20fd-4e6e-902d-a6fd6f8b5466 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:46.110342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c6bce-20fd-4e6e-902d-a6fd6f8b5466 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:46.111486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c6bce-20fd-4e6e-902d-a6fd6f8b5466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:46.112769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c6bce-20fd-4e6e-902d-a6fd6f8b5466 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:46.258051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:46.259002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1172/2353] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:46 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:46.275055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5010795-f561-4882-afb0-4c5463efd35e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:46.399962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc4e34-6a93-4fde-aad9-021a4af2a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:46.400731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc4e34-6a93-4fde-aad9-021a4af2a3ca admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:46.402045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc4e34-6a93-4fde-aad9-021a4af2a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:46.402800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc4e34-6a93-4fde-aad9-021a4af2a3ca admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:46.403917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc4e34-6a93-4fde-aad9-021a4af2a3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:46.409627 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc4e34-6a93-4fde-aad9-021a4af2a3ca admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:46.530897 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1182/2354] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:46 2022] POST /identity/v3/users => generated 287 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:03:46.536131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c6bce-20fd-4e6e-902d-a6fd6f8b5466 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:46.548397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c6bce-20fd-4e6e-902d-a6fd6f8b5466 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:46.553951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988c06c-734d-414d-b3f0-ace6e9fb52b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:46.554771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988c06c-734d-414d-b3f0-ace6e9fb52b0 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:46.555084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988c06c-734d-414d-b3f0-ace6e9fb52b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:46.555352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988c06c-734d-414d-b3f0-ace6e9fb52b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:46.556014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988c06c-734d-414d-b3f0-ace6e9fb52b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:46.559041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988c06c-734d-414d-b3f0-ace6e9fb52b0 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:46.582636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1173/2355] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:46.586591 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc4e34-6a93-4fde-aad9-021a4af2a3ca admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:46.598661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc4e34-6a93-4fde-aad9-021a4af2a3ca admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:46.606722 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b07f1-1f1a-49f4-bf2a-77aa6d37be12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:46.607673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b07f1-1f1a-49f4-bf2a-77aa6d37be12 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:46.612116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b07f1-1f1a-49f4-bf2a-77aa6d37be12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:46.612116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b07f1-1f1a-49f4-bf2a-77aa6d37be12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:46.612116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b07f1-1f1a-49f4-bf2a-77aa6d37be12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:46.612116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b07f1-1f1a-49f4-bf2a-77aa6d37be12 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:46.624138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1183/2356] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:46.628040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988c06c-734d-414d-b3f0-ace6e9fb52b0 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:46.642384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988c06c-734d-414d-b3f0-ace6e9fb52b0 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:46.647324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc032f-6bda-4793-819e-9fa85433fa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:46.648093 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc032f-6bda-4793-819e-9fa85433fa54 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:46.648466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc032f-6bda-4793-819e-9fa85433fa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:46.648833 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc032f-6bda-4793-819e-9fa85433fa54 admin admin] PATH_INFO: `/v3/projects/f5acc35a0352450b9c1b0434978331a4/users/e71e9e4c21ad459182395730734f4f3b/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:46.660595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:46.669154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc032f-6bda-4793-819e-9fa85433fa54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5acc35a0352450b9c1b0434978331a4, user_id=e71e9e4c21ad459182395730734f4f3b, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:46.672314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc032f-6bda-4793-819e-9fa85433fa54 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:46.805305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1174/2357] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:46 2022] PUT /identity/v3/projects/f5acc35a0352450b9c1b0434978331a4/users/e71e9e4c21ad459182395730734f4f3b/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:03:46.878859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8d48-f1ac-4907-9653-1241c81e8de7 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:46.879395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8d48-f1ac-4907-9653-1241c81e8de7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:46.879807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8d48-f1ac-4907-9653-1241c81e8de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:46.916755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-810d8d48-f1ac-4907-9653-1241c81e8de7 None None] MFA Rules not processed for user `e71e9e4c21ad459182395730734f4f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:46.919226 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-810d8d48-f1ac-4907-9653-1241c81e8de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:46.994825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1184/2358] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:03:47.000293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc032f-6bda-4793-819e-9fa85433fa54 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:47.077863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc032f-6bda-4793-819e-9fa85433fa54 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:47.092129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df1020-974f-4b49-85a7-0d2f5297ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:47.093017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1020-974f-4b49-85a7-0d2f5297ca7e admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:47.093302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1020-974f-4b49-85a7-0d2f5297ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:47.093840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1020-974f-4b49-85a7-0d2f5297ca7e admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:47.094478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df1020-974f-4b49-85a7-0d2f5297ca7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:47.096441 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df1020-974f-4b49-85a7-0d2f5297ca7e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:47.229176 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1175/2359] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:46 2022] POST /identity/v3/projects => generated 373 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:47.234935 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d8d48-f1ac-4907-9653-1241c81e8de7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:47.253471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d8d48-f1ac-4907-9653-1241c81e8de7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:47.335414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e43e65-fd11-43aa-a583-595b9e6f319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:47.336408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e43e65-fd11-43aa-a583-595b9e6f319b admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:47.337068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e43e65-fd11-43aa-a583-595b9e6f319b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:47.337589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e43e65-fd11-43aa-a583-595b9e6f319b admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:47.338443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e43e65-fd11-43aa-a583-595b9e6f319b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:47.340735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e43e65-fd11-43aa-a583-595b9e6f319b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:47.349609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:47.464916 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1185/2360] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:47 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:47.468712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df1020-974f-4b49-85a7-0d2f5297ca7e admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:47.505032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df1020-974f-4b49-85a7-0d2f5297ca7e admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:47.528979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d38bf-10b6-4fe8-bfe3-fd12fb0e81a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:47.529658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d38bf-10b6-4fe8-bfe3-fd12fb0e81a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:47.530373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d38bf-10b6-4fe8-bfe3-fd12fb0e81a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:47.530373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d38bf-10b6-4fe8-bfe3-fd12fb0e81a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:47.530893 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d38bf-10b6-4fe8-bfe3-fd12fb0e81a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:47.532602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d38bf-10b6-4fe8-bfe3-fd12fb0e81a2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:47.554973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1176/2361] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:03:47.556435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e43e65-fd11-43aa-a583-595b9e6f319b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:47.570959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e43e65-fd11-43aa-a583-595b9e6f319b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:47.576039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ef965-c1fe-4f20-b5b3-a5927571690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:47.576845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ef965-c1fe-4f20-b5b3-a5927571690c admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:47.577284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ef965-c1fe-4f20-b5b3-a5927571690c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:47.578197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ef965-c1fe-4f20-b5b3-a5927571690c admin admin] PATH_INFO: `/v3/projects/34f9736d058a4f37bf6d27d370809ba4/users/d55ef3325d1846d8824895aac1c38437/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:47.606285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ef965-c1fe-4f20-b5b3-a5927571690c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f9736d058a4f37bf6d27d370809ba4, user_id=d55ef3325d1846d8824895aac1c38437, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:47.615282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ef965-c1fe-4f20-b5b3-a5927571690c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:47.730783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1186/2362] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:47 2022] PUT /identity/v3/projects/34f9736d058a4f37bf6d27d370809ba4/users/d55ef3325d1846d8824895aac1c38437/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:47.731355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d38bf-10b6-4fe8-bfe3-fd12fb0e81a2 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:47.752162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d38bf-10b6-4fe8-bfe3-fd12fb0e81a2 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:47.827895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d12a5-6ed5-47c3-a817-9202037d712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:47.828978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d12a5-6ed5-47c3-a817-9202037d712d admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:47.829436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d12a5-6ed5-47c3-a817-9202037d712d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:47.829869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d12a5-6ed5-47c3-a817-9202037d712d admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:47.830666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d12a5-6ed5-47c3-a817-9202037d712d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:47.832463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d12a5-6ed5-47c3-a817-9202037d712d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:47.845660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1177/2363] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:47.849923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ef965-c1fe-4f20-b5b3-a5927571690c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:47.862633 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ef965-c1fe-4f20-b5b3-a5927571690c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:47.868145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:47.868799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:47.869068 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:47.869331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] PATH_INFO: `/v3/projects/34f9736d058a4f37bf6d27d370809ba4/users/d55ef3325d1846d8824895aac1c38437/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:47.872263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f9736d058a4f37bf6d27d370809ba4, user_id=d55ef3325d1846d8824895aac1c38437, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:47.875079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:47.887655 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1187/2364] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:47 2022] PUT /identity/v3/projects/34f9736d058a4f37bf6d27d370809ba4/users/d55ef3325d1846d8824895aac1c38437/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:47.897927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3677a-d5be-497e-a8f9-4b689c6be7ea None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:47.898202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3677a-d5be-497e-a8f9-4b689c6be7ea None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:47.898472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3677a-d5be-497e-a8f9-4b689c6be7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:47.930329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7cd3677a-d5be-497e-a8f9-4b689c6be7ea None None] MFA Rules not processed for user `d55ef3325d1846d8824895aac1c38437`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:47.931579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7cd3677a-d5be-497e-a8f9-4b689c6be7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:47.987472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:48.017473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1178/2365] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:48.029314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:48.031858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:48.108933 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cc03e-2413-4b4b-aa8c-509b7128fd40 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:48.113748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c2a70-e7e5-4c3f-a045-1d4d56fad2ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:48.114372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2a70-e7e5-4c3f-a045-1d4d56fad2ca service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:48.114837 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2a70-e7e5-4c3f-a045-1d4d56fad2ca service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:48.115043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2a70-e7e5-4c3f-a045-1d4d56fad2ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:48.185630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c2a70-e7e5-4c3f-a045-1d4d56fad2ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:48.187518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c2a70-e7e5-4c3f-a045-1d4d56fad2ca service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:48.200406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1188/2366] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:03:52.262222 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3677a-d5be-497e-a8f9-4b689c6be7ea None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:52.310899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3677a-d5be-497e-a8f9-4b689c6be7ea None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:52.316896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:52.316896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:52.317141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:52.317303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] PATH_INFO: `/v3/users/e71e9e4c21ad459182395730734f4f3b` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:52.318695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71e9e4c21ad459182395730734f4f3b)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:52.320858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:52.897508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71e9e4c21ad459182395730734f4f3b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:52.898343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71e9e4c21ad459182395730734f4f3b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:52.916803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71e9e4c21ad459182395730734f4f3b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:52.917847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71e9e4c21ad459182395730734f4f3b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:53.003268 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71e9e4c21ad459182395730734f4f3b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:53.018034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1179/2367] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:52 2022] DELETE /identity/v3/users/e71e9e4c21ad459182395730734f4f3b => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:53.022592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2073f97-4925-4737-a19a-1e5bbeb356e7 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:53.023273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2073f97-4925-4737-a19a-1e5bbeb356e7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:53.023273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2073f97-4925-4737-a19a-1e5bbeb356e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:53.052842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-c2073f97-4925-4737-a19a-1e5bbeb356e7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:53.054456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c2073f97-4925-4737-a19a-1e5bbeb356e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:53.140471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1189/2368] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:53.155753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:53.157805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:53.223747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373b21f-c905-48fb-8bd4-4f09417bd9e6 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:53.227796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdffab-23f9-414b-9f8c-72ac073c1537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:53.228737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdffab-23f9-414b-9f8c-72ac073c1537 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:53.229281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdffab-23f9-414b-9f8c-72ac073c1537 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:53.229834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdffab-23f9-414b-9f8c-72ac073c1537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:53.240527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cdffab-23f9-414b-9f8c-72ac073c1537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:53.242130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cdffab-23f9-414b-9f8c-72ac073c1537 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:53.253103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1180/2369] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:03:53.724215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2073f97-4925-4737-a19a-1e5bbeb356e7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:53.726026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c2073f97-4925-4737-a19a-1e5bbeb356e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:53.765006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2073f97-4925-4737-a19a-1e5bbeb356e7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:53.771451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:53.777502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:53.777502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:53.777502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] PATH_INFO: `/v3/projects/f5acc35a0352450b9c1b0434978331a4` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:53.777502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5acc35a0352450b9c1b0434978331a4)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:53.777502 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:53.778676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:53.916820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5acc35a0352450b9c1b0434978331a4', 'request_id': 'req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:53.923703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5acc35a0352450b9c1b0434978331a4', 'request_id': 'req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.022043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5acc35a0352450b9c1b0434978331a4', 'request_id': 'req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.034344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] The token cache is being invalidate because project f5acc35a0352450b9c1b0434978331a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:54.036887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.036887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.036887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1190/2370] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:53 2022] DELETE /identity/v3/projects/f5acc35a0352450b9c1b0434978331a4 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:54.041061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdffab-23f9-414b-9f8c-72ac073c1537 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:54.048970 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:54.052306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdffab-23f9-414b-9f8c-72ac073c1537 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:54.111240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:54.111919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:54.112203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:54.112477 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] PATH_INFO: `/v3/users/d55ef3325d1846d8824895aac1c38437` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:54.113660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55ef3325d1846d8824895aac1c38437)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:54.115656 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:54.338632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55ef3325d1846d8824895aac1c38437'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.339255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55ef3325d1846d8824895aac1c38437'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.352045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55ef3325d1846d8824895aac1c38437'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.352677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55ef3325d1846d8824895aac1c38437'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.462373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8c5442ef-1d34-476c-8a40-d9d00b534099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55ef3325d1846d8824895aac1c38437'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:54.476568 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1181/2371] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:03:54 2022] DELETE /identity/v3/users/d55ef3325d1846d8824895aac1c38437 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:03:54.952216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:54.953976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:54.970543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a57b8-c14d-4170-91e9-ff56a0cc4d2f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:55.031310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:55.031940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.032220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.032491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] PATH_INFO: `/v3/projects/34f9736d058a4f37bf6d27d370809ba4` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.033623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f9736d058a4f37bf6d27d370809ba4)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:55.035659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:55.202536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f9736d058a4f37bf6d27d370809ba4', 'request_id': 'req-13da3662-510c-450a-b07f-0cab2aaab75d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:55.208144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f9736d058a4f37bf6d27d370809ba4', 'request_id': 'req-13da3662-510c-450a-b07f-0cab2aaab75d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:55.278780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f9736d058a4f37bf6d27d370809ba4', 'request_id': 'req-13da3662-510c-450a-b07f-0cab2aaab75d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:55.291619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] The token cache is being invalidate because project 34f9736d058a4f37bf6d27d370809ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:03:55.291912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:55.292721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-13da3662-510c-450a-b07f-0cab2aaab75d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:03:55.294400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1191/2372] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:03:54 2022] DELETE /identity/v3/projects/34f9736d058a4f37bf6d27d370809ba4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:55.299344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecd5fb-85c9-4811-ba37-889670ac1f2a None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.299918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecd5fb-85c9-4811-ba37-889670ac1f2a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.299918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecd5fb-85c9-4811-ba37-889670ac1f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.321076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-63ecd5fb-85c9-4811-ba37-889670ac1f2a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:55.322293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-63ecd5fb-85c9-4811-ba37-889670ac1f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:55.383571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1182/2373] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:03:55.393827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022139-cd4c-482e-aad6-d98523860617 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.394122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022139-cd4c-482e-aad6-d98523860617 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.394399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022139-cd4c-482e-aad6-d98523860617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.414727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e2022139-cd4c-482e-aad6-d98523860617 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:55.415974 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e2022139-cd4c-482e-aad6-d98523860617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:55.424186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1192/2374] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:55.428387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecd5fb-85c9-4811-ba37-889670ac1f2a None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:55.439392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecd5fb-85c9-4811-ba37-889670ac1f2a None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:55.444691 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:55.445289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.445564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.445951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.446599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:03:55.446901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:55.448530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:55.461503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1183/2375] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:03:55.466566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2022139-cd4c-482e-aad6-d98523860617 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:55.476100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2022139-cd4c-482e-aad6-d98523860617 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:55.481596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af5ba4-cc54-44b7-849f-37a914aa2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:55.482197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af5ba4-cc54-44b7-849f-37a914aa2c3f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.482474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af5ba4-cc54-44b7-849f-37a914aa2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.482738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af5ba4-cc54-44b7-849f-37a914aa2c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.483412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af5ba4-cc54-44b7-849f-37a914aa2c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:55.485334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af5ba4-cc54-44b7-849f-37a914aa2c3f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:55.497386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:55.603125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1193/2376] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:55 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:55.608614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:55.624553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88017b-e1fc-4799-b95f-f2c13537e5e9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:55.633090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:55.683768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e04069-c5d4-4d8b-8d55-cd3e348b3954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:55.684335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e04069-c5d4-4d8b-8d55-cd3e348b3954 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.684608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e04069-c5d4-4d8b-8d55-cd3e348b3954 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.684873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e04069-c5d4-4d8b-8d55-cd3e348b3954 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.685487 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e04069-c5d4-4d8b-8d55-cd3e348b3954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:55.687432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e04069-c5d4-4d8b-8d55-cd3e348b3954 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:55.828099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1184/2377] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:55 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:55.832285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af5ba4-cc54-44b7-849f-37a914aa2c3f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:55.843483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af5ba4-cc54-44b7-849f-37a914aa2c3f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:55.848002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5ccc2-36b0-44e6-afad-a75d76f5bfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:55.848596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5ccc2-36b0-44e6-afad-a75d76f5bfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.848868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5ccc2-36b0-44e6-afad-a75d76f5bfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.849137 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5ccc2-36b0-44e6-afad-a75d76f5bfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.849764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5ccc2-36b0-44e6-afad-a75d76f5bfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:55.851464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5ccc2-36b0-44e6-afad-a75d76f5bfc1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:55.862173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1194/2378] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:55.866443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e04069-c5d4-4d8b-8d55-cd3e348b3954 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:55.877617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e04069-c5d4-4d8b-8d55-cd3e348b3954 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:55.882442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd1877-cdf2-4410-8e03-b83d745a7e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:55.887088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1877-cdf2-4410-8e03-b83d745a7e3e admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.887088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1877-cdf2-4410-8e03-b83d745a7e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.887088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1877-cdf2-4410-8e03-b83d745a7e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.887088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd1877-cdf2-4410-8e03-b83d745a7e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:55.887088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd1877-cdf2-4410-8e03-b83d745a7e3e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:55.897514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1185/2379] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:55.901742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5ccc2-36b0-44e6-afad-a75d76f5bfc1 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:55.912738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5ccc2-36b0-44e6-afad-a75d76f5bfc1 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:55.917495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f36df-ac64-446e-aa7b-3cc5b2437f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:55.924972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f36df-ac64-446e-aa7b-3cc5b2437f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:55.924972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f36df-ac64-446e-aa7b-3cc5b2437f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:55.924972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f36df-ac64-446e-aa7b-3cc5b2437f0b admin admin] PATH_INFO: `/v3/projects/935173665bc54928b903b748597200c3/users/8787d48bf4c642f69186a1eb05f19cd4/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:55.935217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f36df-ac64-446e-aa7b-3cc5b2437f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=935173665bc54928b903b748597200c3, user_id=8787d48bf4c642f69186a1eb05f19cd4, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:55.937936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f36df-ac64-446e-aa7b-3cc5b2437f0b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:56.050119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1195/2380] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:55 2022] PUT /identity/v3/projects/935173665bc54928b903b748597200c3/users/8787d48bf4c642f69186a1eb05f19cd4/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:56.061375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c521c50-d512-45da-96ce-f3ce65a7a6a7 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:56.062082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c521c50-d512-45da-96ce-f3ce65a7a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:56.062082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c521c50-d512-45da-96ce-f3ce65a7a6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:56.097261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6c521c50-d512-45da-96ce-f3ce65a7a6a7 None None] MFA Rules not processed for user `8787d48bf4c642f69186a1eb05f19cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:56.098509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6c521c50-d512-45da-96ce-f3ce65a7a6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:56.174979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1186/2381] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:03:56.185965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f36df-ac64-446e-aa7b-3cc5b2437f0b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:56.256285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f36df-ac64-446e-aa7b-3cc5b2437f0b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:56.260991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d1d46-117e-444e-aa10-c5994642d53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:56.261620 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d1d46-117e-444e-aa10-c5994642d53f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:56.261901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d1d46-117e-444e-aa10-c5994642d53f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:56.262166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d1d46-117e-444e-aa10-c5994642d53f admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:56.262832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d1d46-117e-444e-aa10-c5994642d53f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:56.264876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d1d46-117e-444e-aa10-c5994642d53f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:56.389023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1196/2382] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:56 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:56.393065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c521c50-d512-45da-96ce-f3ce65a7a6a7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:56.404489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c521c50-d512-45da-96ce-f3ce65a7a6a7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:56.464398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e7696-36c9-41e3-8802-fde93aad3baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:56.464951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e7696-36c9-41e3-8802-fde93aad3baf admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:56.465232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e7696-36c9-41e3-8802-fde93aad3baf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:56.465501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e7696-36c9-41e3-8802-fde93aad3baf admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:56.466130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e7696-36c9-41e3-8802-fde93aad3baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:56.468212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e7696-36c9-41e3-8802-fde93aad3baf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:56.566856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1187/2383] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:56 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:56.572595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d1d46-117e-444e-aa10-c5994642d53f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:56.583728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d1d46-117e-444e-aa10-c5994642d53f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:56.587870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc772-a5b0-4abf-bffd-c9c6e805e50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:56.588464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc772-a5b0-4abf-bffd-c9c6e805e50e admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:56.588735 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc772-a5b0-4abf-bffd-c9c6e805e50e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:56.589001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc772-a5b0-4abf-bffd-c9c6e805e50e admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:56.590174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cc772-a5b0-4abf-bffd-c9c6e805e50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:56.591796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cc772-a5b0-4abf-bffd-c9c6e805e50e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:56.602468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1197/2384] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:56.607175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e7696-36c9-41e3-8802-fde93aad3baf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:56.617276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e7696-36c9-41e3-8802-fde93aad3baf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:56.621926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783e0a-9e57-4a87-90a8-25f365bd3f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:56.622519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783e0a-9e57-4a87-90a8-25f365bd3f24 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:56.622796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783e0a-9e57-4a87-90a8-25f365bd3f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:56.623110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783e0a-9e57-4a87-90a8-25f365bd3f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:56.623750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5783e0a-9e57-4a87-90a8-25f365bd3f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:56.625366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5783e0a-9e57-4a87-90a8-25f365bd3f24 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:56.627448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:56.635486 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1188/2385] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:56.643745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc772-a5b0-4abf-bffd-c9c6e805e50e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:56.654607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc772-a5b0-4abf-bffd-c9c6e805e50e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:56.659506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f0b9b-c95e-462c-b2fe-30a6e0dd25b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:56.660348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f0b9b-c95e-462c-b2fe-30a6e0dd25b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:56.660478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f0b9b-c95e-462c-b2fe-30a6e0dd25b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:56.660712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f0b9b-c95e-462c-b2fe-30a6e0dd25b8 admin admin] PATH_INFO: `/v3/projects/656b289f19be450a87242ceebe7a5485/users/16f35f9a53a7496f9579a43ffdf7f45f/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:56.676797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f0b9b-c95e-462c-b2fe-30a6e0dd25b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=656b289f19be450a87242ceebe7a5485, user_id=16f35f9a53a7496f9579a43ffdf7f45f, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:56.679721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f0b9b-c95e-462c-b2fe-30a6e0dd25b8 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:56.770688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1198/2386] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:56 2022] PUT /identity/v3/projects/656b289f19be450a87242ceebe7a5485/users/16f35f9a53a7496f9579a43ffdf7f45f/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:03:56.781693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573233-ebbb-4176-82d6-cbbd19593e99 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:56.781979 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573233-ebbb-4176-82d6-cbbd19593e99 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:56.782256 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573233-ebbb-4176-82d6-cbbd19593e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:56.811369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-9f573233-ebbb-4176-82d6-cbbd19593e99 None None] MFA Rules not processed for user `16f35f9a53a7496f9579a43ffdf7f45f`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:56.812603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9f573233-ebbb-4176-82d6-cbbd19593e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:56.880757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1189/2387] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:56.885411 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f0b9b-c95e-462c-b2fe-30a6e0dd25b8 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:56.952489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f0b9b-c95e-462c-b2fe-30a6e0dd25b8 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:56.958330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146009ab-82cd-4d62-8e68-f8c3d35caa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:56.959213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146009ab-82cd-4d62-8e68-f8c3d35caa2d admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:56.959500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146009ab-82cd-4d62-8e68-f8c3d35caa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:56.959772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146009ab-82cd-4d62-8e68-f8c3d35caa2d admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:56.960433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146009ab-82cd-4d62-8e68-f8c3d35caa2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:56.962383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146009ab-82cd-4d62-8e68-f8c3d35caa2d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:56.975951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:57.089357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1199/2388] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:03:56 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:03:57.090437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f573233-ebbb-4176-82d6-cbbd19593e99 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:57.101516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f573233-ebbb-4176-82d6-cbbd19593e99 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:57.165817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128a9db-e8cc-4ef3-bb80-e00a5749bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:57.166454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128a9db-e8cc-4ef3-bb80-e00a5749bd3c admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:57.166725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128a9db-e8cc-4ef3-bb80-e00a5749bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:57.167021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128a9db-e8cc-4ef3-bb80-e00a5749bd3c admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:57.168307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128a9db-e8cc-4ef3-bb80-e00a5749bd3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:57.170372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128a9db-e8cc-4ef3-bb80-e00a5749bd3c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:57.304898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1190/2389] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:03:57 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:03:57.306677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146009ab-82cd-4d62-8e68-f8c3d35caa2d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:57.318184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146009ab-82cd-4d62-8e68-f8c3d35caa2d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:57.322750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8482b-926b-4a89-ac0a-056d1a08d778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:57.323381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8482b-926b-4a89-ac0a-056d1a08d778 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:57.323661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8482b-926b-4a89-ac0a-056d1a08d778 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:57.324105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8482b-926b-4a89-ac0a-056d1a08d778 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:57.324962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8482b-926b-4a89-ac0a-056d1a08d778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:57.326543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8482b-926b-4a89-ac0a-056d1a08d778 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:57.336554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1200/2390] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:57.341493 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128a9db-e8cc-4ef3-bb80-e00a5749bd3c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:57.351836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128a9db-e8cc-4ef3-bb80-e00a5749bd3c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:57.357028 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de7c6-7eb6-464c-9ccb-b178a0a20f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:57.357689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de7c6-7eb6-464c-9ccb-b178a0a20f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:57.357960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de7c6-7eb6-464c-9ccb-b178a0a20f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:57.358226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de7c6-7eb6-464c-9ccb-b178a0a20f49 admin admin] PATH_INFO: `/v3/projects/b2765c8d241040b28f17bc23c839b0c2/users/da6accb17cf140d6adf6456daa11d8d3/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:57.377257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109de7c6-7eb6-464c-9ccb-b178a0a20f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2765c8d241040b28f17bc23c839b0c2, user_id=da6accb17cf140d6adf6456daa11d8d3, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:57.378804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109de7c6-7eb6-464c-9ccb-b178a0a20f49 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:57.510426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1191/2391] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:57 2022] PUT /identity/v3/projects/b2765c8d241040b28f17bc23c839b0c2/users/da6accb17cf140d6adf6456daa11d8d3/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:57.514420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8482b-926b-4a89-ac0a-056d1a08d778 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:57.524663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8482b-926b-4a89-ac0a-056d1a08d778 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:57.583488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be462491-59ef-408d-80ab-2c575eb005a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:57.584114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be462491-59ef-408d-80ab-2c575eb005a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:57.584393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be462491-59ef-408d-80ab-2c575eb005a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:57.584663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be462491-59ef-408d-80ab-2c575eb005a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:57.585304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be462491-59ef-408d-80ab-2c575eb005a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:57.586896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be462491-59ef-408d-80ab-2c575eb005a0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:57.597736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1201/2392] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:03:57.602091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de7c6-7eb6-464c-9ccb-b178a0a20f49 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:57.614081 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de7c6-7eb6-464c-9ccb-b178a0a20f49 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:57.618739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:57.619433 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:57.619697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:57.619966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] PATH_INFO: `/v3/projects/b2765c8d241040b28f17bc23c839b0c2/users/da6accb17cf140d6adf6456daa11d8d3/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:57.622178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2765c8d241040b28f17bc23c839b0c2, user_id=da6accb17cf140d6adf6456daa11d8d3, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:57.624822 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:57.634896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1192/2393] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:03:57 2022] PUT /identity/v3/projects/b2765c8d241040b28f17bc23c839b0c2/users/da6accb17cf140d6adf6456daa11d8d3/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:03:57.646713 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f47680-b84d-40b9-966e-c26b2b20babd None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:57.647037 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f47680-b84d-40b9-966e-c26b2b20babd None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:57.647314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f47680-b84d-40b9-966e-c26b2b20babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:57.676013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a9f47680-b84d-40b9-966e-c26b2b20babd None None] MFA Rules not processed for user `da6accb17cf140d6adf6456daa11d8d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:03:57.677261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a9f47680-b84d-40b9-966e-c26b2b20babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:57.748544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1202/2394] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:03:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:03:57.759989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:03:57.762652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:03:57.806985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:03:57.820761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f7076-705a-43c8-8767-8932fbc52b42 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:03:57.825298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:03:57.825888 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:03:57.826163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:03:57.826431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:03:57.883040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:03:57.884473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:03:57.894488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1193/2395] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:03:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:00.087447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f47680-b84d-40b9-966e-c26b2b20babd None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:00.098371 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f47680-b84d-40b9-966e-c26b2b20babd None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:00.101982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddc70e-2b4d-4034-bb51-25b5174e04e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:00.102580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddc70e-2b4d-4034-bb51-25b5174e04e7 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:00.102850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddc70e-2b4d-4034-bb51-25b5174e04e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:00.103253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddc70e-2b4d-4034-bb51-25b5174e04e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:00.162209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddc70e-2b4d-4034-bb51-25b5174e04e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:00.163756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddc70e-2b4d-4034-bb51-25b5174e04e7 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:00.169745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:00.174888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1203/2396] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:01.795541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:01.796960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:01.808415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df587c5-ecb5-471a-ba35-4983045c83f2 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:01.812533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48487d86-abdc-4696-a5ed-3d090be9877c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:01.813133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487d86-abdc-4696-a5ed-3d090be9877c service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:01.813410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487d86-abdc-4696-a5ed-3d090be9877c service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:01.813688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48487d86-abdc-4696-a5ed-3d090be9877c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:01.823206 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48487d86-abdc-4696-a5ed-3d090be9877c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:01.825133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48487d86-abdc-4696-a5ed-3d090be9877c service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:01.838834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1194/2397] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:06.760561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddc70e-2b4d-4034-bb51-25b5174e04e7 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:06.832571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddc70e-2b4d-4034-bb51-25b5174e04e7 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:06.843738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:06.844398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:06.844673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:06.844943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] PATH_INFO: `/v3/users/8787d48bf4c642f69186a1eb05f19cd4` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:06.846106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8787d48bf4c642f69186a1eb05f19cd4)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:06.848106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:07.118840 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8787d48bf4c642f69186a1eb05f19cd4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:07.119663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8787d48bf4c642f69186a1eb05f19cd4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:07.136877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8787d48bf4c642f69186a1eb05f19cd4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:07.253662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8787d48bf4c642f69186a1eb05f19cd4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:07.272775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8787d48bf4c642f69186a1eb05f19cd4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:07.288727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1204/2398] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:06 2022] DELETE /identity/v3/users/8787d48bf4c642f69186a1eb05f19cd4 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:07.293070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9dcd3e-0f97-4b25-8cd4-bf66312b9bb2 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:07.294087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9dcd3e-0f97-4b25-8cd4-bf66312b9bb2 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:07.295749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9dcd3e-0f97-4b25-8cd4-bf66312b9bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:07.337399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4c9dcd3e-0f97-4b25-8cd4-bf66312b9bb2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:07.339024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4c9dcd3e-0f97-4b25-8cd4-bf66312b9bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:07.426874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1195/2399] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:07.516917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:07.518789 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:07.648301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48caa2b-7b60-4284-923a-6c73f1ab79ee admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:07.656809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:07.657456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:07.658167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:07.658167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:07.668769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:07.670637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:07.682823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1205/2400] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:07.914548 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85cf92-d2e1-417d-8249-e913a3e65e49 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:07.915141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85cf92-d2e1-417d-8249-e913a3e65e49 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:07.915572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85cf92-d2e1-417d-8249-e913a3e65e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:07.941712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ae85cf92-d2e1-417d-8249-e913a3e65e49 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:07.941712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ae85cf92-d2e1-417d-8249-e913a3e65e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:08.032449 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1196/2401] 10.222.0.94 () {58 vars in 1009 bytes} [Fri Aug 26 16:04:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:08.048895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:08.048895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:08.081527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:08.138966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36b903-b668-4bbc-a44f-558fe178d808 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:08.144894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b60ff5-7998-4df9-9357-fb1c3d79a1cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:08.145526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b60ff5-7998-4df9-9357-fb1c3d79a1cf service swift] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:08.145798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b60ff5-7998-4df9-9357-fb1c3d79a1cf service swift] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:08.146064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b60ff5-7998-4df9-9357-fb1c3d79a1cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:08.156125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b60ff5-7998-4df9-9357-fb1c3d79a1cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:08.157767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b60ff5-7998-4df9-9357-fb1c3d79a1cf service swift] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:08.167380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1206/2402] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:04:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:04:08.335847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85cf92-d2e1-417d-8249-e913a3e65e49 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:08.337677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ae85cf92-d2e1-417d-8249-e913a3e65e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:08.355558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85cf92-d2e1-417d-8249-e913a3e65e49 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:08.361973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:08.362828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:08.363433 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:08.363630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] PATH_INFO: `/v3/projects/935173665bc54928b903b748597200c3` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:08.364878 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935173665bc54928b903b748597200c3)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:08.366740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:08.436285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:08.573063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935173665bc54928b903b748597200c3', 'request_id': 'req-ad69ad4a-e372-4b81-8806-c99d6c54aa69'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:08.585808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935173665bc54928b903b748597200c3', 'request_id': 'req-ad69ad4a-e372-4b81-8806-c99d6c54aa69'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:08.678787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935173665bc54928b903b748597200c3', 'request_id': 'req-ad69ad4a-e372-4b81-8806-c99d6c54aa69'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:08.701980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] The token cache is being invalidate because project 935173665bc54928b903b748597200c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:08.702484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:08.703379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:08.705490 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1197/2403] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:08 2022] DELETE /identity/v3/projects/935173665bc54928b903b748597200c3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:08.709550 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b60ff5-7998-4df9-9357-fb1c3d79a1cf service swift] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:08.723826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b60ff5-7998-4df9-9357-fb1c3d79a1cf service swift] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:08.853300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:08.854609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:08.855372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:08.855959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] PATH_INFO: `/v3/users/16f35f9a53a7496f9579a43ffdf7f45f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:08.858829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f35f9a53a7496f9579a43ffdf7f45f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:08.862556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:09.133848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f35f9a53a7496f9579a43ffdf7f45f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:09.134549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f35f9a53a7496f9579a43ffdf7f45f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:09.148955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f35f9a53a7496f9579a43ffdf7f45f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:09.238661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f35f9a53a7496f9579a43ffdf7f45f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:09.246259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f35f9a53a7496f9579a43ffdf7f45f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:09.253716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1207/2404] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:08 2022] DELETE /identity/v3/users/16f35f9a53a7496f9579a43ffdf7f45f => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:09.762863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:09.764565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:09.791502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69ad4a-e372-4b81-8806-c99d6c54aa69 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:09.864111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:09.865060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:09.865397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:09.865776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] PATH_INFO: `/v3/projects/656b289f19be450a87242ceebe7a5485` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:09.867072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=656b289f19be450a87242ceebe7a5485)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:09.869526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:10.037805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '656b289f19be450a87242ceebe7a5485', 'request_id': 'req-d0173668-6179-4291-b3bc-dcc6371c2897'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:10.049013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '656b289f19be450a87242ceebe7a5485', 'request_id': 'req-d0173668-6179-4291-b3bc-dcc6371c2897'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:10.176514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '656b289f19be450a87242ceebe7a5485', 'request_id': 'req-d0173668-6179-4291-b3bc-dcc6371c2897'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:10.278299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] The token cache is being invalidate because project 656b289f19be450a87242ceebe7a5485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:10.278299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:10.278299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:10.278299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1198/2405] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:09 2022] DELETE /identity/v3/projects/656b289f19be450a87242ceebe7a5485 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:10.278967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:10.278967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:10.278967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1328a20-7c7c-44e6-a05f-79fd29b7f104 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:10.278967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:10.280219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:10.281001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:10.281223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:10.281498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] PATH_INFO: `/v3/users/da6accb17cf140d6adf6456daa11d8d3` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:10.282719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6accb17cf140d6adf6456daa11d8d3)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:10.284718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:11.053850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6accb17cf140d6adf6456daa11d8d3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:11.054904 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6accb17cf140d6adf6456daa11d8d3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:11.070382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6accb17cf140d6adf6456daa11d8d3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:11.141854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6accb17cf140d6adf6456daa11d8d3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:11.155313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-565ce04e-8268-43c8-8358-e134d7afb0d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6accb17cf140d6adf6456daa11d8d3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:11.164005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1208/2406] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:10 2022] DELETE /identity/v3/users/da6accb17cf140d6adf6456daa11d8d3 => generated 0 bytes in 966 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:11.700143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:11.703274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:11.717992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0173668-6179-4291-b3bc-dcc6371c2897 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:11.767643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:11.790136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:11.790748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:11.791095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:11.791411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] PATH_INFO: `/v3/projects/b2765c8d241040b28f17bc23c839b0c2` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:11.792612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2765c8d241040b28f17bc23c839b0c2)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:11.794876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:11.998732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2765c8d241040b28f17bc23c839b0c2', 'request_id': 'req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:12.005004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2765c8d241040b28f17bc23c839b0c2', 'request_id': 'req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:12.157459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2765c8d241040b28f17bc23c839b0c2', 'request_id': 'req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:12.174530 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] The token cache is being invalidate because project b2765c8d241040b28f17bc23c839b0c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:12.176871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:12.177734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-721b6b52-ac75-4ffd-98e7-c4d4acdad61c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:12.179795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1199/2407] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:11 2022] DELETE /identity/v3/projects/b2765c8d241040b28f17bc23c839b0c2 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:12.185029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165587b8-9213-442c-ace2-c51e03b8f067 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:12.185586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165587b8-9213-442c-ace2-c51e03b8f067 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:12.185586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165587b8-9213-442c-ace2-c51e03b8f067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:12.212352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-165587b8-9213-442c-ace2-c51e03b8f067 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:12.215596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-165587b8-9213-442c-ace2-c51e03b8f067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:12.296812 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2207f5-6604-4af2-a112-6630c2c40429 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:12.297517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2207f5-6604-4af2-a112-6630c2c40429 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:12.297789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2207f5-6604-4af2-a112-6630c2c40429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:12.306937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1209/2408] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:12.338871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6e2207f5-6604-4af2-a112-6630c2c40429 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:12.340380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6e2207f5-6604-4af2-a112-6630c2c40429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:12.350277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1200/2409] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:12.354815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165587b8-9213-442c-ace2-c51e03b8f067 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:12.383340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165587b8-9213-442c-ace2-c51e03b8f067 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:12.396793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:12.397885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:12.398271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:12.398628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:12.410316 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:04:12.410316 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:12.410316 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:12.421982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1210/2410] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:04:12.428445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2207f5-6604-4af2-a112-6630c2c40429 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:12.457687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2207f5-6604-4af2-a112-6630c2c40429 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:12.463075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadd21e-ffdf-40c0-a545-ea23ada15264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:12.463705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadd21e-ffdf-40c0-a545-ea23ada15264 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:12.464011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadd21e-ffdf-40c0-a545-ea23ada15264 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:12.464293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadd21e-ffdf-40c0-a545-ea23ada15264 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:12.464976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadd21e-ffdf-40c0-a545-ea23ada15264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:12.467071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadd21e-ffdf-40c0-a545-ea23ada15264 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:12.587381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1201/2411] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:12 2022] POST /identity/v3/projects => generated 399 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:12.592762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:12.607909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104f8ef-73fd-4225-aa37-bc4bed23b94e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:12.653610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:12.687547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fab03c-107d-4f85-83ab-63eeb54e8318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:12.688529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fab03c-107d-4f85-83ab-63eeb54e8318 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:12.691337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fab03c-107d-4f85-83ab-63eeb54e8318 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:12.692074 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fab03c-107d-4f85-83ab-63eeb54e8318 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:12.693303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fab03c-107d-4f85-83ab-63eeb54e8318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:12.695910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fab03c-107d-4f85-83ab-63eeb54e8318 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:12.836535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1211/2412] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:12 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:12.840567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadd21e-ffdf-40c0-a545-ea23ada15264 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:12.851740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadd21e-ffdf-40c0-a545-ea23ada15264 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:12.858090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd28ee-2e14-4a92-9940-b4d819916459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:12.858984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd28ee-2e14-4a92-9940-b4d819916459 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:12.859359 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd28ee-2e14-4a92-9940-b4d819916459 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:12.859725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd28ee-2e14-4a92-9940-b4d819916459 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:12.860593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd28ee-2e14-4a92-9940-b4d819916459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:12.862563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd28ee-2e14-4a92-9940-b4d819916459 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:12.876075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1202/2413] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:12.884115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fab03c-107d-4f85-83ab-63eeb54e8318 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:12.895178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fab03c-107d-4f85-83ab-63eeb54e8318 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:12.900081 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43280d-ede9-475b-ac95-09ccf486bb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:12.900830 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43280d-ede9-475b-ac95-09ccf486bb9a admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:12.901118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43280d-ede9-475b-ac95-09ccf486bb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:12.901400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43280d-ede9-475b-ac95-09ccf486bb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:12.902237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43280d-ede9-475b-ac95-09ccf486bb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:12.904261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43280d-ede9-475b-ac95-09ccf486bb9a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:12.918101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1212/2414] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:04:12.924169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd28ee-2e14-4a92-9940-b4d819916459 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:12.935987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd28ee-2e14-4a92-9940-b4d819916459 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:12.941168 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:12.941813 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:12.942115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:12.942645 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] PATH_INFO: `/v3/projects/e2b497d0cb4548249248341063060b33/users/afe4fc8de21142dabe87545eb6669ffd/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:12.959854 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b497d0cb4548249248341063060b33, user_id=afe4fc8de21142dabe87545eb6669ffd, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:12.962715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:13.087762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1203/2415] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:12 2022] PUT /identity/v3/projects/e2b497d0cb4548249248341063060b33/users/afe4fc8de21142dabe87545eb6669ffd/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:13.100406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fa71b-d94d-4355-8c28-33a7facb0c02 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:13.100691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fa71b-d94d-4355-8c28-33a7facb0c02 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:13.100972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fa71b-d94d-4355-8c28-33a7facb0c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:13.132920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4b5fa71b-d94d-4355-8c28-33a7facb0c02 None None] MFA Rules not processed for user `afe4fc8de21142dabe87545eb6669ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:13.134220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4b5fa71b-d94d-4355-8c28-33a7facb0c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:13.220476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1213/2416] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:13.233660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:13.235389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:13.305245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378f20-e3fc-4eb0-a1e5-28905ae4d2db admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:13.309180 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3885bc-8b41-43de-b201-9329cb385109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:13.309961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3885bc-8b41-43de-b201-9329cb385109 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:13.310366 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3885bc-8b41-43de-b201-9329cb385109 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:13.310642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3885bc-8b41-43de-b201-9329cb385109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:13.320250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3885bc-8b41-43de-b201-9329cb385109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:13.321729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3885bc-8b41-43de-b201-9329cb385109 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:13.331580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1204/2417] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:13 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:23.215870 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:23.327551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:23.699206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fa71b-d94d-4355-8c28-33a7facb0c02 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:23.766046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fa71b-d94d-4355-8c28-33a7facb0c02 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:23.770532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:23.771197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:23.771474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:23.771746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] PATH_INFO: `/v3/users/afe4fc8de21142dabe87545eb6669ffd` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:23.772855 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe4fc8de21142dabe87545eb6669ffd)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:23.774778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:24.055093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe4fc8de21142dabe87545eb6669ffd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:24.055719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe4fc8de21142dabe87545eb6669ffd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:24.068714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe4fc8de21142dabe87545eb6669ffd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:24.204143 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe4fc8de21142dabe87545eb6669ffd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:24.212080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe4fc8de21142dabe87545eb6669ffd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:24.219256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1214/2418] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:23 2022] DELETE /identity/v3/users/afe4fc8de21142dabe87545eb6669ffd => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:24.223794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce9605-0009-4aaa-8afe-e63e1d5674a1 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:24.224107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce9605-0009-4aaa-8afe-e63e1d5674a1 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:24.224384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce9605-0009-4aaa-8afe-e63e1d5674a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:24.244745 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-a2ce9605-0009-4aaa-8afe-e63e1d5674a1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:24.250638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a2ce9605-0009-4aaa-8afe-e63e1d5674a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:24.319350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1205/2419] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:24.321284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:24.322804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:24.385306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a5e70-0ef0-45a9-9598-285a9c3aceb6 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:24.389157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c21e5-3cb7-45de-94b3-a57557f94584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:24.389882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c21e5-3cb7-45de-94b3-a57557f94584 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:24.390159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c21e5-3cb7-45de-94b3-a57557f94584 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:24.390432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c21e5-3cb7-45de-94b3-a57557f94584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:24.400286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c21e5-3cb7-45de-94b3-a57557f94584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:24.401733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c21e5-3cb7-45de-94b3-a57557f94584 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:24.412814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1215/2420] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:25.046079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce9605-0009-4aaa-8afe-e63e1d5674a1 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:25.047741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a2ce9605-0009-4aaa-8afe-e63e1d5674a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:25.065611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce9605-0009-4aaa-8afe-e63e1d5674a1 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:25.071252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:25.072046 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:25.072274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:25.072541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] PATH_INFO: `/v3/projects/e2b497d0cb4548249248341063060b33` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:25.073830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b497d0cb4548249248341063060b33)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:25.075797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:25.332412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b497d0cb4548249248341063060b33', 'request_id': 'req-818a8861-2df9-45d9-a7de-db55433a6f00'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:25.337694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b497d0cb4548249248341063060b33', 'request_id': 'req-818a8861-2df9-45d9-a7de-db55433a6f00'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:25.463268 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b497d0cb4548249248341063060b33', 'request_id': 'req-818a8861-2df9-45d9-a7de-db55433a6f00'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:25.475039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] The token cache is being invalidate because project e2b497d0cb4548249248341063060b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:25.481037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:25.481037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:25.481037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1206/2421] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:25 2022] DELETE /identity/v3/projects/e2b497d0cb4548249248341063060b33 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:27.464060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc28ee-a35c-452e-a3d9-24bb57fbb8d7 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:27.464624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc28ee-a35c-452e-a3d9-24bb57fbb8d7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:27.464624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc28ee-a35c-452e-a3d9-24bb57fbb8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:27.491783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e1bc28ee-a35c-452e-a3d9-24bb57fbb8d7 None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:27.493017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e1bc28ee-a35c-452e-a3d9-24bb57fbb8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:27.514482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:27.553170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1216/2422] 10.222.0.94 () {58 vars in 967 bytes} [Fri Aug 26 16:04:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:27.569581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:27.571558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:27.651603 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8861-2df9-45d9-a7de-db55433a6f00 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:27.653700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157af3e-664c-4cc5-b930-bb5762a7df4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:27.654320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157af3e-664c-4cc5-b930-bb5762a7df4c service swift] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:27.654585 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157af3e-664c-4cc5-b930-bb5762a7df4c service swift] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:27.654853 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157af3e-664c-4cc5-b930-bb5762a7df4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:27.664232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157af3e-664c-4cc5-b930-bb5762a7df4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:27.668350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157af3e-664c-4cc5-b930-bb5762a7df4c service swift] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:27.676406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1207/2423] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:04:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:04:27.716149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc28ee-a35c-452e-a3d9-24bb57fbb8d7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:27.719908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e1bc28ee-a35c-452e-a3d9-24bb57fbb8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:27.778414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc28ee-a35c-452e-a3d9-24bb57fbb8d7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:27.782375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:27.783022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:27.783302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:27.783571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:27.785346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:27.850888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:27.852376 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:27.862832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1217/2424] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:31.245798 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673b0b-2ac0-4fba-8cef-1f9209eed6ac None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:31.246314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673b0b-2ac0-4fba-8cef-1f9209eed6ac None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:31.246314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673b0b-2ac0-4fba-8cef-1f9209eed6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:31.266852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-35673b0b-2ac0-4fba-8cef-1f9209eed6ac None None] MFA Rules not processed for user `9e1dd86804944fd1806ec289181b361e`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:31.268096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-35673b0b-2ac0-4fba-8cef-1f9209eed6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:31.276892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1208/2425] 10.222.0.94 () {58 vars in 967 bytes} [Fri Aug 26 16:04:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:31.290103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:31.302571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9e467-e8df-410a-8205-be7f5489d3bf service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:31.307755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015b4c9-262c-43f0-a0f3-deb3ed272fff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b92a2d92e27d4fe48b42f9e04be92372', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:31.308432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015b4c9-262c-43f0-a0f3-deb3ed272fff service swift] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:31.308647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015b4c9-262c-43f0-a0f3-deb3ed272fff service swift] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:31.308912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015b4c9-262c-43f0-a0f3-deb3ed272fff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:31.318505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015b4c9-262c-43f0-a0f3-deb3ed272fff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:31.320111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015b4c9-262c-43f0-a0f3-deb3ed272fff service swift] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:31.332210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1218/2426] 10.222.0.94 () {60 vars in 1384 bytes} [Fri Aug 26 16:04:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 16:04:32.210787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673b0b-2ac0-4fba-8cef-1f9209eed6ac None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:32.212733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-35673b0b-2ac0-4fba-8cef-1f9209eed6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:32.231561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673b0b-2ac0-4fba-8cef-1f9209eed6ac None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:32.234481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:32.235211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:32.235482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:32.235744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] PATH_INFO: `/v3/users/7069a9b54a3245b3ab9671730344e6d3` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:32.237067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7069a9b54a3245b3ab9671730344e6d3)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:32.239070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:32.240773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:32.443436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7069a9b54a3245b3ab9671730344e6d3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:32.444072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7069a9b54a3245b3ab9671730344e6d3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:32.455771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7069a9b54a3245b3ab9671730344e6d3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:32.456394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7069a9b54a3245b3ab9671730344e6d3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:32.577559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c614f885-4577-472a-9c1f-69eee4acc01d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7069a9b54a3245b3ab9671730344e6d3'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:32.591274 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1209/2427] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:32 2022] DELETE /identity/v3/users/7069a9b54a3245b3ab9671730344e6d3 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:32.817268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015b4c9-262c-43f0-a0f3-deb3ed272fff service swift] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:32.828359 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015b4c9-262c-43f0-a0f3-deb3ed272fff service swift] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:32.893372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:32.894007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:32.894282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:32.894556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] PATH_INFO: `/v3/projects/8c69c9fb12124d72954b8ab7ad024175` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:32.895711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c69c9fb12124d72954b8ab7ad024175)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:32.897616 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:32.904374 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:33.033273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c69c9fb12124d72954b8ab7ad024175', 'request_id': 'req-eb65cd75-4025-43fa-a432-61ef901d197c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:33.039631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c69c9fb12124d72954b8ab7ad024175', 'request_id': 'req-eb65cd75-4025-43fa-a432-61ef901d197c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:33.126098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c69c9fb12124d72954b8ab7ad024175', 'request_id': 'req-eb65cd75-4025-43fa-a432-61ef901d197c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:33.141105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] The token cache is being invalidate because project 8c69c9fb12124d72954b8ab7ad024175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:33.141105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:33.141105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eb65cd75-4025-43fa-a432-61ef901d197c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:33.141497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1219/2428] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:32 2022] DELETE /identity/v3/projects/8c69c9fb12124d72954b8ab7ad024175 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:33.146550 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759092b-17e9-4ab8-a2ba-e0ddabb37935 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:33.146871 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759092b-17e9-4ab8-a2ba-e0ddabb37935 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:33.147169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759092b-17e9-4ab8-a2ba-e0ddabb37935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:33.169077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f759092b-17e9-4ab8-a2ba-e0ddabb37935 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:33.170280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f759092b-17e9-4ab8-a2ba-e0ddabb37935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:33.259847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1210/2429] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:33.269918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0996c3-307b-43d4-a2c7-fe74eb86749a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:33.270205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0996c3-307b-43d4-a2c7-fe74eb86749a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:33.270472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0996c3-307b-43d4-a2c7-fe74eb86749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:33.290801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5c0996c3-307b-43d4-a2c7-fe74eb86749a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:33.291972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5c0996c3-307b-43d4-a2c7-fe74eb86749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:33.300384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1220/2430] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:33.304993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759092b-17e9-4ab8-a2ba-e0ddabb37935 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:33.316916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759092b-17e9-4ab8-a2ba-e0ddabb37935 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:33.322129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:33.322754 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:33.323070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:33.323342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:33.324006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:04:33.324307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:33.325995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:33.342973 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1211/2431] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:04:33.344174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0996c3-307b-43d4-a2c7-fe74eb86749a None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:33.354058 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0996c3-307b-43d4-a2c7-fe74eb86749a None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:33.359167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd10d1-ff58-4d1f-88cc-f761978efcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:33.359790 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd10d1-ff58-4d1f-88cc-f761978efcef admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:33.360064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd10d1-ff58-4d1f-88cc-f761978efcef admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:33.360327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd10d1-ff58-4d1f-88cc-f761978efcef admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:33.360953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd10d1-ff58-4d1f-88cc-f761978efcef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:33.362858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd10d1-ff58-4d1f-88cc-f761978efcef admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:33.534873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1221/2432] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:33 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:33.538891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:33.558197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca0d07-8831-4a0e-afd3-a57beec66689 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:33.625712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4962b2-16f5-47ad-8a88-8c2d9d407b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:33.626332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4962b2-16f5-47ad-8a88-8c2d9d407b43 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:33.626613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4962b2-16f5-47ad-8a88-8c2d9d407b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:33.626883 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4962b2-16f5-47ad-8a88-8c2d9d407b43 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:33.627557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4962b2-16f5-47ad-8a88-8c2d9d407b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:33.629480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4962b2-16f5-47ad-8a88-8c2d9d407b43 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:33.641634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:33.715229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1212/2433] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:33 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:33.716459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd10d1-ff58-4d1f-88cc-f761978efcef admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:33.727849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd10d1-ff58-4d1f-88cc-f761978efcef admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:33.732756 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff428476-3310-4115-8bbb-473e4d5e51fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:33.733639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428476-3310-4115-8bbb-473e4d5e51fb admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:33.733918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428476-3310-4115-8bbb-473e4d5e51fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:33.734183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428476-3310-4115-8bbb-473e4d5e51fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:33.739039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff428476-3310-4115-8bbb-473e4d5e51fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:33.739039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff428476-3310-4115-8bbb-473e4d5e51fb admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:33.747490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1222/2434] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:33.751611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4962b2-16f5-47ad-8a88-8c2d9d407b43 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:33.762546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4962b2-16f5-47ad-8a88-8c2d9d407b43 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:33.767904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eda778-4fba-4450-9151-b2ac613622e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:33.768517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eda778-4fba-4450-9151-b2ac613622e7 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:33.768827 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eda778-4fba-4450-9151-b2ac613622e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:33.769129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eda778-4fba-4450-9151-b2ac613622e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:33.769773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eda778-4fba-4450-9151-b2ac613622e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:33.771459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eda778-4fba-4450-9151-b2ac613622e7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:33.783593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1213/2435] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:33.796527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff428476-3310-4115-8bbb-473e4d5e51fb admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:33.808167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff428476-3310-4115-8bbb-473e4d5e51fb admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:33.813097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20698f-b926-48d6-98ab-7b5dc69cb607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:33.813760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20698f-b926-48d6-98ab-7b5dc69cb607 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:33.814030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20698f-b926-48d6-98ab-7b5dc69cb607 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:33.814297 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20698f-b926-48d6-98ab-7b5dc69cb607 admin admin] PATH_INFO: `/v3/projects/5ac003f333c843878317629bfe72c26d/users/06d36c0b2837464e97e2641cfe36f3c1/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:33.832242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20698f-b926-48d6-98ab-7b5dc69cb607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac003f333c843878317629bfe72c26d, user_id=06d36c0b2837464e97e2641cfe36f3c1, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:33.842603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20698f-b926-48d6-98ab-7b5dc69cb607 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:34.030261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1223/2436] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:33 2022] PUT /identity/v3/projects/5ac003f333c843878317629bfe72c26d/users/06d36c0b2837464e97e2641cfe36f3c1/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:34.042392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc0aa-9476-48a2-a56d-81e570025940 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:34.042392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc0aa-9476-48a2-a56d-81e570025940 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:34.042392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc0aa-9476-48a2-a56d-81e570025940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:34.083880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5a6bc0aa-9476-48a2-a56d-81e570025940 None None] MFA Rules not processed for user `06d36c0b2837464e97e2641cfe36f3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:34.085126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5a6bc0aa-9476-48a2-a56d-81e570025940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:34.174880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1214/2437] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:34.179736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20698f-b926-48d6-98ab-7b5dc69cb607 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:34.300195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:34.316238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20698f-b926-48d6-98ab-7b5dc69cb607 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:34.321957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e2e41-21e6-40a6-bcf9-3ec062854d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:34.323567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e2e41-21e6-40a6-bcf9-3ec062854d57 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:34.323567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e2e41-21e6-40a6-bcf9-3ec062854d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:34.323567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e2e41-21e6-40a6-bcf9-3ec062854d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:34.324775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e2e41-21e6-40a6-bcf9-3ec062854d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:34.326614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e2e41-21e6-40a6-bcf9-3ec062854d57 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:34.470990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1224/2438] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:34 2022] POST /identity/v3/projects => generated 371 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:34.475124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bc0aa-9476-48a2-a56d-81e570025940 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:34.503948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bc0aa-9476-48a2-a56d-81e570025940 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:34.574258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12037dca-829f-48a3-9662-9c3b5b36f6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:34.575039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12037dca-829f-48a3-9662-9c3b5b36f6bf admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:34.581397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12037dca-829f-48a3-9662-9c3b5b36f6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:34.581397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12037dca-829f-48a3-9662-9c3b5b36f6bf admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:34.581397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12037dca-829f-48a3-9662-9c3b5b36f6bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:34.581397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12037dca-829f-48a3-9662-9c3b5b36f6bf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:34.672774 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1215/2439] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:34 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:34.676751 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e2e41-21e6-40a6-bcf9-3ec062854d57 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:34.687228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e2e41-21e6-40a6-bcf9-3ec062854d57 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:34.692675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b13a3-bcf2-42b1-927c-bd77d0079332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:34.693392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b13a3-bcf2-42b1-927c-bd77d0079332 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:34.693667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b13a3-bcf2-42b1-927c-bd77d0079332 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:34.703605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b13a3-bcf2-42b1-927c-bd77d0079332 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:34.703605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b13a3-bcf2-42b1-927c-bd77d0079332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:34.703605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b13a3-bcf2-42b1-927c-bd77d0079332 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:34.710417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1225/2440] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:34.714449 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12037dca-829f-48a3-9662-9c3b5b36f6bf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:34.727713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12037dca-829f-48a3-9662-9c3b5b36f6bf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:34.733509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735df4b-1687-4651-b218-e2efe2eb14da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:34.734192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735df4b-1687-4651-b218-e2efe2eb14da admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:34.734471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735df4b-1687-4651-b218-e2efe2eb14da admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:34.734744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735df4b-1687-4651-b218-e2efe2eb14da admin admin] PATH_INFO: `/v3/projects/cbfbd3ed4ad8479980afe0b293c736ef/users/2a4660c7311a43158ad41873a8a09fa4/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:34.753111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735df4b-1687-4651-b218-e2efe2eb14da admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfbd3ed4ad8479980afe0b293c736ef, user_id=2a4660c7311a43158ad41873a8a09fa4, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:34.755903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735df4b-1687-4651-b218-e2efe2eb14da admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:34.856244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1216/2441] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:34 2022] PUT /identity/v3/projects/cbfbd3ed4ad8479980afe0b293c736ef/users/2a4660c7311a43158ad41873a8a09fa4/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:34.861172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b13a3-bcf2-42b1-927c-bd77d0079332 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:34.872211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b13a3-bcf2-42b1-927c-bd77d0079332 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:34.931320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462508c-7753-4f87-b9db-6e336326f1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:34.931934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462508c-7753-4f87-b9db-6e336326f1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:34.932206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462508c-7753-4f87-b9db-6e336326f1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:34.932474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462508c-7753-4f87-b9db-6e336326f1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:34.933115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462508c-7753-4f87-b9db-6e336326f1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:34.934703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462508c-7753-4f87-b9db-6e336326f1a6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:34.945813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1226/2442] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:34.949682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735df4b-1687-4651-b218-e2efe2eb14da admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:34.961621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735df4b-1687-4651-b218-e2efe2eb14da admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:34.968786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:34.969463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:34.969738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:34.970049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] PATH_INFO: `/v3/projects/cbfbd3ed4ad8479980afe0b293c736ef/users/2a4660c7311a43158ad41873a8a09fa4/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:34.972413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfbd3ed4ad8479980afe0b293c736ef, user_id=2a4660c7311a43158ad41873a8a09fa4, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:34.975095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:34.989755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1217/2443] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:34 2022] PUT /identity/v3/projects/cbfbd3ed4ad8479980afe0b293c736ef/users/2a4660c7311a43158ad41873a8a09fa4/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:35.000515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579049c-1660-44fe-9337-e35974bac988 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:35.000799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579049c-1660-44fe-9337-e35974bac988 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:35.001070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579049c-1660-44fe-9337-e35974bac988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:35.032066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-e579049c-1660-44fe-9337-e35974bac988 None None] MFA Rules not processed for user `2a4660c7311a43158ad41873a8a09fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:35.034065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e579049c-1660-44fe-9337-e35974bac988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:35.121259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1227/2444] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:35.133371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:35.135256 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:35.148844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:35.223167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6be32-84ca-4278-aa65-edb81150ac43 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:35.232269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c37bd9-a12d-47a0-b955-92b901f40547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:35.233127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c37bd9-a12d-47a0-b955-92b901f40547 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:35.233994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c37bd9-a12d-47a0-b955-92b901f40547 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:35.234723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c37bd9-a12d-47a0-b955-92b901f40547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:35.245988 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c37bd9-a12d-47a0-b955-92b901f40547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:35.247475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c37bd9-a12d-47a0-b955-92b901f40547 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:35.258099 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1218/2445] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:36.606273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579049c-1660-44fe-9337-e35974bac988 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:36.641605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:36.679063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579049c-1660-44fe-9337-e35974bac988 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:36.683719 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:36.684342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:36.684621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:36.684892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] PATH_INFO: `/v3/users/06d36c0b2837464e97e2641cfe36f3c1` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:36.686027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d36c0b2837464e97e2641cfe36f3c1)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:36.688022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:36.969492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d36c0b2837464e97e2641cfe36f3c1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:36.970842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d36c0b2837464e97e2641cfe36f3c1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:36.993912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d36c0b2837464e97e2641cfe36f3c1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:37.108764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d36c0b2837464e97e2641cfe36f3c1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:37.117770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d36c0b2837464e97e2641cfe36f3c1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:37.125302 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1228/2446] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:36 2022] DELETE /identity/v3/users/06d36c0b2837464e97e2641cfe36f3c1 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:37.129968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51607e7e-6806-488c-b0f4-ec82df447e81 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:37.130444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51607e7e-6806-488c-b0f4-ec82df447e81 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:37.130566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51607e7e-6806-488c-b0f4-ec82df447e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:37.162986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-51607e7e-6806-488c-b0f4-ec82df447e81 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:37.164210 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-51607e7e-6806-488c-b0f4-ec82df447e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:37.231563 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1219/2447] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:37.241624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:37.244602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:37.306117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f689b2-1d17-4283-9206-03fe08079a1a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:37.309821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027231d-f2e4-4ac9-8388-67b092468fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:37.310429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027231d-f2e4-4ac9-8388-67b092468fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:37.310702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027231d-f2e4-4ac9-8388-67b092468fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:37.310994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027231d-f2e4-4ac9-8388-67b092468fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:37.321605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027231d-f2e4-4ac9-8388-67b092468fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:37.323238 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027231d-f2e4-4ac9-8388-67b092468fe9 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:37.333991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1229/2448] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:37.899185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51607e7e-6806-488c-b0f4-ec82df447e81 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:37.900654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-51607e7e-6806-488c-b0f4-ec82df447e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:37.913335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51607e7e-6806-488c-b0f4-ec82df447e81 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:37.918278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:37.918869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:37.919182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:37.919450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] PATH_INFO: `/v3/projects/5ac003f333c843878317629bfe72c26d` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:37.920631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac003f333c843878317629bfe72c26d)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:37.923156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:38.127240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac003f333c843878317629bfe72c26d', 'request_id': 'req-26744997-d3fa-42b9-b665-4f499fb4534b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.132977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac003f333c843878317629bfe72c26d', 'request_id': 'req-26744997-d3fa-42b9-b665-4f499fb4534b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.244144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac003f333c843878317629bfe72c26d', 'request_id': 'req-26744997-d3fa-42b9-b665-4f499fb4534b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.263782 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] The token cache is being invalidate because project 5ac003f333c843878317629bfe72c26d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:38.265253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.265445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.269939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1220/2449] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:37 2022] DELETE /identity/v3/projects/5ac003f333c843878317629bfe72c26d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:38.271186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027231d-f2e4-4ac9-8388-67b092468fe9 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:38.284810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027231d-f2e4-4ac9-8388-67b092468fe9 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:38.351793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:38.372684 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:38.379084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:38.379084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:38.379084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] PATH_INFO: `/v3/users/2a4660c7311a43158ad41873a8a09fa4` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:38.379084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4660c7311a43158ad41873a8a09fa4)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:38.379084 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:38.671712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4660c7311a43158ad41873a8a09fa4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.672480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4660c7311a43158ad41873a8a09fa4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.701085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4660c7311a43158ad41873a8a09fa4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.843873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4660c7311a43158ad41873a8a09fa4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.853508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-830ed287-ee51-49a3-b135-aaa8f3968da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4660c7311a43158ad41873a8a09fa4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:38.861988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1230/2450] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:38 2022] DELETE /identity/v3/users/2a4660c7311a43158ad41873a8a09fa4 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:04:39.457617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:39.459665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:39.474734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744997-d3fa-42b9-b665-4f499fb4534b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:39.494415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:39.593982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:39.594821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:39.595248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:39.595766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] PATH_INFO: `/v3/projects/cbfbd3ed4ad8479980afe0b293c736ef` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:39.599626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfbd3ed4ad8479980afe0b293c736ef)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:39.602149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:39.752573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfbd3ed4ad8479980afe0b293c736ef', 'request_id': 'req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:39.758165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfbd3ed4ad8479980afe0b293c736ef', 'request_id': 'req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:39.868431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfbd3ed4ad8479980afe0b293c736ef', 'request_id': 'req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:39.882151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] The token cache is being invalidate because project cbfbd3ed4ad8479980afe0b293c736ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:39.882443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:39.883277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-cd667a8b-0d48-4e0c-b1c6-c0ac1f20323d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:39.884930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1221/2451] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:39 2022] DELETE /identity/v3/projects/cbfbd3ed4ad8479980afe0b293c736ef => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:39.889952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d120e-a544-49ca-8046-0b62d067db30 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:39.890460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d120e-a544-49ca-8046-0b62d067db30 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:39.890460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d120e-a544-49ca-8046-0b62d067db30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:39.946566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-381d120e-a544-49ca-8046-0b62d067db30 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:39.953463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-381d120e-a544-49ca-8046-0b62d067db30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:40.026859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1231/2452] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:40.037087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427711-afb2-4576-8c12-328c4fe932b1 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:40.037508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427711-afb2-4576-8c12-328c4fe932b1 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:40.037847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427711-afb2-4576-8c12-328c4fe932b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:40.060640 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-86427711-afb2-4576-8c12-328c4fe932b1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:40.062784 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-86427711-afb2-4576-8c12-328c4fe932b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:40.074705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1222/2453] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:40.075954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d120e-a544-49ca-8046-0b62d067db30 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:40.087493 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d120e-a544-49ca-8046-0b62d067db30 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:40.092559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:40.093318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:40.094058 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:40.094397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:40.095167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:04:40.095541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:40.097292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:40.110724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1232/2454] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:04:40.114906 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86427711-afb2-4576-8c12-328c4fe932b1 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:40.126132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86427711-afb2-4576-8c12-328c4fe932b1 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:40.132555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb15b1-3802-4153-ab74-af50add49059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:40.133910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb15b1-3802-4153-ab74-af50add49059 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:40.134259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb15b1-3802-4153-ab74-af50add49059 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:40.134587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb15b1-3802-4153-ab74-af50add49059 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:40.140131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb15b1-3802-4153-ab74-af50add49059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:40.140131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb15b1-3802-4153-ab74-af50add49059 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:40.380118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1223/2455] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:40 2022] POST /identity/v3/projects => generated 365 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:40.384406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:40.405432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ed5b-72bc-4473-ad7d-4a30ad381ca7 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:40.441335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:40.478965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b815047-0b67-4b7d-a255-8b0e4b8d678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:40.479597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b815047-0b67-4b7d-a255-8b0e4b8d678c admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:40.479881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b815047-0b67-4b7d-a255-8b0e4b8d678c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:40.480155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b815047-0b67-4b7d-a255-8b0e4b8d678c admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:40.481546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b815047-0b67-4b7d-a255-8b0e4b8d678c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:40.484147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b815047-0b67-4b7d-a255-8b0e4b8d678c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:40.597150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1233/2456] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:40 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:40.601257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb15b1-3802-4153-ab74-af50add49059 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:40.612726 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb15b1-3802-4153-ab74-af50add49059 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:40.617559 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec06d8-dcdf-47d0-9cda-a16794cc6557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:40.618164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec06d8-dcdf-47d0-9cda-a16794cc6557 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:40.618444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec06d8-dcdf-47d0-9cda-a16794cc6557 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:40.618715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec06d8-dcdf-47d0-9cda-a16794cc6557 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:40.619389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec06d8-dcdf-47d0-9cda-a16794cc6557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:40.621007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec06d8-dcdf-47d0-9cda-a16794cc6557 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:40.632386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1224/2457] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:04:40.636387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b815047-0b67-4b7d-a255-8b0e4b8d678c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:40.647442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b815047-0b67-4b7d-a255-8b0e4b8d678c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:40.652006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e578f-b628-4b5c-bf91-edd1ce2a62fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:40.652611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e578f-b628-4b5c-bf91-edd1ce2a62fa admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:40.652873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e578f-b628-4b5c-bf91-edd1ce2a62fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:40.653138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e578f-b628-4b5c-bf91-edd1ce2a62fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:40.653770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e578f-b628-4b5c-bf91-edd1ce2a62fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:40.657111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e578f-b628-4b5c-bf91-edd1ce2a62fa admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:40.667383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1234/2458] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:40.671972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec06d8-dcdf-47d0-9cda-a16794cc6557 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:40.683088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec06d8-dcdf-47d0-9cda-a16794cc6557 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:40.687918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491ab45-b952-4a9b-a0ca-f80ae11cbf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:40.688553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ab45-b952-4a9b-a0ca-f80ae11cbf7a admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:40.688825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ab45-b952-4a9b-a0ca-f80ae11cbf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:40.689091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ab45-b952-4a9b-a0ca-f80ae11cbf7a admin admin] PATH_INFO: `/v3/projects/eddbc5d2a8fa42dd80fe4190ffaf45b2/users/9e979a86760e493c9b712625fff6fd20/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:40.704085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491ab45-b952-4a9b-a0ca-f80ae11cbf7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddbc5d2a8fa42dd80fe4190ffaf45b2, user_id=9e979a86760e493c9b712625fff6fd20, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:40.706751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491ab45-b952-4a9b-a0ca-f80ae11cbf7a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:40.807376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1225/2459] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:40 2022] PUT /identity/v3/projects/eddbc5d2a8fa42dd80fe4190ffaf45b2/users/9e979a86760e493c9b712625fff6fd20/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:40.823895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88016323-12a9-4c46-9910-dfe381b669f6 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:40.824207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88016323-12a9-4c46-9910-dfe381b669f6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:40.824448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88016323-12a9-4c46-9910-dfe381b669f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:40.885086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-88016323-12a9-4c46-9910-dfe381b669f6 None None] MFA Rules not processed for user `9e979a86760e493c9b712625fff6fd20`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:40.887130 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-88016323-12a9-4c46-9910-dfe381b669f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:41.020504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1235/2460] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:41.024661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491ab45-b952-4a9b-a0ca-f80ae11cbf7a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:41.097476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:41.132337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491ab45-b952-4a9b-a0ca-f80ae11cbf7a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:41.140964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c64a71-b5eb-489c-9bd2-dd2de2c44927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:41.141609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c64a71-b5eb-489c-9bd2-dd2de2c44927 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:41.141882 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c64a71-b5eb-489c-9bd2-dd2de2c44927 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:41.142164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c64a71-b5eb-489c-9bd2-dd2de2c44927 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:41.142799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c64a71-b5eb-489c-9bd2-dd2de2c44927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:41.145237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c64a71-b5eb-489c-9bd2-dd2de2c44927 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:41.313318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1226/2461] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:41 2022] POST /identity/v3/projects => generated 365 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:41.315291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88016323-12a9-4c46-9910-dfe381b669f6 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:41.337235 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88016323-12a9-4c46-9910-dfe381b669f6 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:41.396856 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd930f3-6617-4d17-b53e-f0dc0b7d89f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:41.397446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd930f3-6617-4d17-b53e-f0dc0b7d89f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:41.397725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd930f3-6617-4d17-b53e-f0dc0b7d89f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:41.397996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd930f3-6617-4d17-b53e-f0dc0b7d89f8 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:41.398631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd930f3-6617-4d17-b53e-f0dc0b7d89f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:41.400603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd930f3-6617-4d17-b53e-f0dc0b7d89f8 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:41.559330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1236/2462] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:41 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:41.563677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c64a71-b5eb-489c-9bd2-dd2de2c44927 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:41.574612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c64a71-b5eb-489c-9bd2-dd2de2c44927 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:41.580432 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:41.581052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:41.581315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:41.581591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:41.582217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:41.583897 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:41.594319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1227/2463] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:04:41.598805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd930f3-6617-4d17-b53e-f0dc0b7d89f8 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:41.609718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd930f3-6617-4d17-b53e-f0dc0b7d89f8 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:41.614443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995e088-375a-40e7-8807-69f482afa6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:41.615218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995e088-375a-40e7-8807-69f482afa6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:41.615564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995e088-375a-40e7-8807-69f482afa6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:41.615903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995e088-375a-40e7-8807-69f482afa6bf admin admin] PATH_INFO: `/v3/projects/5cd75844078d45da8e68e90d53e70ab0/users/b1fd05a3abc54c8ba7edaa5325bb578c/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:41.632398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995e088-375a-40e7-8807-69f482afa6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd75844078d45da8e68e90d53e70ab0, user_id=b1fd05a3abc54c8ba7edaa5325bb578c, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:41.635176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995e088-375a-40e7-8807-69f482afa6bf admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:41.667686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:41.669480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:41.684003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c7651-d604-454c-ac51-18d6ba89239a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:41.689489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:41.690127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:41.690404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:41.690727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] PATH_INFO: `/v3/users/36d61fd7e97d433e886aaf495f703751` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:41.692167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d61fd7e97d433e886aaf495f703751)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:41.694121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:41.755224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1237/2464] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:41 2022] PUT /identity/v3/projects/5cd75844078d45da8e68e90d53e70ab0/users/b1fd05a3abc54c8ba7edaa5325bb578c/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:41.759184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995e088-375a-40e7-8807-69f482afa6bf admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:41.763588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:41.823690 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995e088-375a-40e7-8807-69f482afa6bf admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:41.828955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12e300-f341-45a1-b325-8522b9599df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:41.829554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12e300-f341-45a1-b325-8522b9599df6 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:41.829857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12e300-f341-45a1-b325-8522b9599df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:41.830126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12e300-f341-45a1-b325-8522b9599df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:41.830763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12e300-f341-45a1-b325-8522b9599df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:41.832399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12e300-f341-45a1-b325-8522b9599df6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:41.842152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1238/2465] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:04:41.846607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12e300-f341-45a1-b325-8522b9599df6 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:41.856142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12e300-f341-45a1-b325-8522b9599df6 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:41.861574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f0bbb-4650-4afd-aa8b-4bc5f5d864c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:41.862232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f0bbb-4650-4afd-aa8b-4bc5f5d864c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:41.862511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f0bbb-4650-4afd-aa8b-4bc5f5d864c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:41.862783 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f0bbb-4650-4afd-aa8b-4bc5f5d864c2 admin admin] PATH_INFO: `/v3/projects/5cd75844078d45da8e68e90d53e70ab0/users/b1fd05a3abc54c8ba7edaa5325bb578c/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:41.864725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f0bbb-4650-4afd-aa8b-4bc5f5d864c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd75844078d45da8e68e90d53e70ab0, user_id=b1fd05a3abc54c8ba7edaa5325bb578c, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:41.867387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f0bbb-4650-4afd-aa8b-4bc5f5d864c2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:41.878133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1239/2466] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:41 2022] PUT /identity/v3/projects/5cd75844078d45da8e68e90d53e70ab0/users/b1fd05a3abc54c8ba7edaa5325bb578c/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:04:41.888988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e214e-937d-4a73-a9ae-09e1726555cf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:41.889296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e214e-937d-4a73-a9ae-09e1726555cf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:41.889570 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e214e-937d-4a73-a9ae-09e1726555cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:41.908569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d61fd7e97d433e886aaf495f703751'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:41.909250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d61fd7e97d433e886aaf495f703751'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:41.917955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3b9e214e-937d-4a73-a9ae-09e1726555cf None None] MFA Rules not processed for user `b1fd05a3abc54c8ba7edaa5325bb578c`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:41.919266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3b9e214e-937d-4a73-a9ae-09e1726555cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:41.921997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d61fd7e97d433e886aaf495f703751'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:41.922684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d61fd7e97d433e886aaf495f703751'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:41.988705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1240/2467] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:04:41.988921 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d61fd7e97d433e886aaf495f703751'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:42.000710 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e214e-937d-4a73-a9ae-09e1726555cf None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:42.002065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3b9e214e-937d-4a73-a9ae-09e1726555cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:42.003614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1228/2468] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:41 2022] DELETE /identity/v3/users/36d61fd7e97d433e886aaf495f703751 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:04:42.060842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e214e-937d-4a73-a9ae-09e1726555cf None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:42.122915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577aa38-a324-4219-813d-6f0c5902f46f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:42.124827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577aa38-a324-4219-813d-6f0c5902f46f service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:42.124827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577aa38-a324-4219-813d-6f0c5902f46f service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:42.124984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577aa38-a324-4219-813d-6f0c5902f46f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:42.199819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577aa38-a324-4219-813d-6f0c5902f46f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:42.203062 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577aa38-a324-4219-813d-6f0c5902f46f service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:42.211101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:42.215912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1241/2469] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:04:42.297663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:42.299391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:42.408576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf496-1d4b-49d4-8dbf-edaf57a6e2ce admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:42.409933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:42.410574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:42.410851 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:42.411161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] PATH_INFO: `/v3/projects/070e0f497888488d863e2f63b87a31ff` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:42.412348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=070e0f497888488d863e2f63b87a31ff)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:42.414316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:42.528177 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:42.548608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '070e0f497888488d863e2f63b87a31ff', 'request_id': 'req-9a920f45-b731-45f7-868c-422810217534'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:42.554615 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '070e0f497888488d863e2f63b87a31ff', 'request_id': 'req-9a920f45-b731-45f7-868c-422810217534'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:42.645247 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '070e0f497888488d863e2f63b87a31ff', 'request_id': 'req-9a920f45-b731-45f7-868c-422810217534'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:42.658103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] The token cache is being invalidate because project 070e0f497888488d863e2f63b87a31ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:42.658394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:42.659080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:42.660579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1229/2470] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:42 2022] DELETE /identity/v3/projects/070e0f497888488d863e2f63b87a31ff => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:42.669588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577aa38-a324-4219-813d-6f0c5902f46f service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:42.680491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577aa38-a324-4219-813d-6f0c5902f46f service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:42.740758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:42.741390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:42.741663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:42.741929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] PATH_INFO: `/v3/users/ee66a17a796c4d0ebfbc1d5779c97450` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:42.743098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee66a17a796c4d0ebfbc1d5779c97450)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:42.745017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:43.002221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee66a17a796c4d0ebfbc1d5779c97450'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.002891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee66a17a796c4d0ebfbc1d5779c97450'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.022996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee66a17a796c4d0ebfbc1d5779c97450'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.129400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee66a17a796c4d0ebfbc1d5779c97450'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.137331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:43.141320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee66a17a796c4d0ebfbc1d5779c97450'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.169539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1242/2471] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:42 2022] DELETE /identity/v3/users/ee66a17a796c4d0ebfbc1d5779c97450 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:43.286593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:43.288856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:43.306651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920f45-b731-45f7-868c-422810217534 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:43.378416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:43.379479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:43.379479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:43.379852 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] PATH_INFO: `/v3/users/9e979a86760e493c9b712625fff6fd20` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:43.382443 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e979a86760e493c9b712625fff6fd20)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:43.384660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:43.672697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e979a86760e493c9b712625fff6fd20'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.673599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e979a86760e493c9b712625fff6fd20'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.690207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e979a86760e493c9b712625fff6fd20'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.690964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e979a86760e493c9b712625fff6fd20'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.802786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:43.804560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:43.831541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5d47-63c1-43b9-923a-564651bcccac admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:43.838305 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:43.839240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:43.839669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:43.840125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] PATH_INFO: `/v3/projects/2e294fd9a4bd4fea914485eec964748b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:43.840850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0f1b4b4d-10bb-4b5e-a4e5-a4bcc069e2d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e979a86760e493c9b712625fff6fd20'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:43.841900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e294fd9a4bd4fea914485eec964748b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:43.852857 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:43.870760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1230/2472] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:43 2022] DELETE /identity/v3/users/9e979a86760e493c9b712625fff6fd20 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:43.875859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a4978-d261-4f08-b20d-fa0dbb37ff95 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:43.876439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a4978-d261-4f08-b20d-fa0dbb37ff95 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:43.876722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a4978-d261-4f08-b20d-fa0dbb37ff95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:43.899016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-373a4978-d261-4f08-b20d-fa0dbb37ff95 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:43.904643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-373a4978-d261-4f08-b20d-fa0dbb37ff95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:43.940253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:43.983027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1231/2473] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:43.993315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a4978-d261-4f08-b20d-fa0dbb37ff95 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:43.994982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-373a4978-d261-4f08-b20d-fa0dbb37ff95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:44.015026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e294fd9a4bd4fea914485eec964748b', 'request_id': 'req-4337e4a1-3f92-4bf2-a5f6-27188916bb90'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:44.023053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e294fd9a4bd4fea914485eec964748b', 'request_id': 'req-4337e4a1-3f92-4bf2-a5f6-27188916bb90'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:44.076626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a4978-d261-4f08-b20d-fa0dbb37ff95 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:44.134545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8be2f-1b92-4d3e-a2fb-e3a29093139d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:44.135333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8be2f-1b92-4d3e-a2fb-e3a29093139d service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.135684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8be2f-1b92-4d3e-a2fb-e3a29093139d service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.136025 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8be2f-1b92-4d3e-a2fb-e3a29093139d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.180843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e294fd9a4bd4fea914485eec964748b', 'request_id': 'req-4337e4a1-3f92-4bf2-a5f6-27188916bb90'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:44.193795 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] The token cache is being invalidate because project 2e294fd9a4bd4fea914485eec964748b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:44.194354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:44.195030 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4337e4a1-3f92-4bf2-a5f6-27188916bb90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:44.196700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1243/2474] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:43 2022] DELETE /identity/v3/projects/2e294fd9a4bd4fea914485eec964748b => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:04:44.201739 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bc25e-d98b-4bd5-9e3a-6c3a0c308cf5 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.202145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bc25e-d98b-4bd5-9e3a-6c3a0c308cf5 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.202536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bc25e-d98b-4bd5-9e3a-6c3a0c308cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.209959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8be2f-1b92-4d3e-a2fb-e3a29093139d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:44.211885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8be2f-1b92-4d3e-a2fb-e3a29093139d service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:44.227005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1232/2475] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:44.231274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b22bc25e-d98b-4bd5-9e3a-6c3a0c308cf5 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:44.233189 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b22bc25e-d98b-4bd5-9e3a-6c3a0c308cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:44.241917 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1244/2476] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:04:44.316155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4de88e-bfb8-4e32-9c75-4fd576780390 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.316928 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4de88e-bfb8-4e32-9c75-4fd576780390 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.317030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4de88e-bfb8-4e32-9c75-4fd576780390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.338384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0a4de88e-bfb8-4e32-9c75-4fd576780390 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:44.339647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0a4de88e-bfb8-4e32-9c75-4fd576780390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:44.350236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1233/2477] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:44.354164 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bc25e-d98b-4bd5-9e3a-6c3a0c308cf5 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:44.364190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bc25e-d98b-4bd5-9e3a-6c3a0c308cf5 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:44.368759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:44.369364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.369709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.369902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.370545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:04:44.370841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:44.372449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:44.384736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1245/2478] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:04:44.388914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4de88e-bfb8-4e32-9c75-4fd576780390 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:44.398999 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4de88e-bfb8-4e32-9c75-4fd576780390 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:44.406622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:44.407481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.407913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.408188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.408951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:44.410721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:44.504952 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1234/2479] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:44 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:44.508913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:44.523752 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60843-6028-4163-94ea-13e42097a852 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:44.605909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63804b5-7a09-4311-8626-4ab01e556e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:44.606522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63804b5-7a09-4311-8626-4ab01e556e08 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.606796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63804b5-7a09-4311-8626-4ab01e556e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.607093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63804b5-7a09-4311-8626-4ab01e556e08 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.607823 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63804b5-7a09-4311-8626-4ab01e556e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:44.609885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63804b5-7a09-4311-8626-4ab01e556e08 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:44.738341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1246/2480] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:44 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:44.738457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:44.739009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:44.740575 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63804b5-7a09-4311-8626-4ab01e556e08 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:44.752408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63804b5-7a09-4311-8626-4ab01e556e08 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:44.758458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73373c6-b278-4d68-a835-463b3a064797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:44.759175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73373c6-b278-4d68-a835-463b3a064797 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.759446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73373c6-b278-4d68-a835-463b3a064797 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.759952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73373c6-b278-4d68-a835-463b3a064797 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.760923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73373c6-b278-4d68-a835-463b3a064797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:44.762851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73373c6-b278-4d68-a835-463b3a064797 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:44.767180 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2651d-7754-4de3-8a81-c7661b4aab9b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:44.781714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:44.782387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.782527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1247/2481] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:44.782723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.783034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] PATH_INFO: `/v3/projects/eddbc5d2a8fa42dd80fe4190ffaf45b2` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.783639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73373c6-b278-4d68-a835-463b3a064797 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:44.784345 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=eddbc5d2a8fa42dd80fe4190ffaf45b2)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:44.786385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:44.809100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73373c6-b278-4d68-a835-463b3a064797 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:44.830043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d51f3-5a2a-44e6-b9d2-99f73c310e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:44.830043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d51f3-5a2a-44e6-b9d2-99f73c310e4d admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.830043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d51f3-5a2a-44e6-b9d2-99f73c310e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.830043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d51f3-5a2a-44e6-b9d2-99f73c310e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.830500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d51f3-5a2a-44e6-b9d2-99f73c310e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:44.830500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d51f3-5a2a-44e6-b9d2-99f73c310e4d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:44.841395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1248/2482] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:44.845858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d51f3-5a2a-44e6-b9d2-99f73c310e4d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:44.857646 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d51f3-5a2a-44e6-b9d2-99f73c310e4d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:44.863165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:44.863940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:44.864333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:44.864676 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] PATH_INFO: `/v3/projects/3c5f0e0c02e64047b9ca17a7320b0bf5/users/c911a9324cb744a6a1aeb068565d3f89/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:44.886352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5f0e0c02e64047b9ca17a7320b0bf5, user_id=c911a9324cb744a6a1aeb068565d3f89, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:44.890177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:44.987563 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1249/2483] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:44 2022] PUT /identity/v3/projects/3c5f0e0c02e64047b9ca17a7320b0bf5/users/c911a9324cb744a6a1aeb068565d3f89/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:04:44.992203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:44.994256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:44.997609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eddbc5d2a8fa42dd80fe4190ffaf45b2', 'request_id': 'req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:44.998394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:45.006488 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eddbc5d2a8fa42dd80fe4190ffaf45b2', 'request_id': 'req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.028794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:45.083614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eddbc5d2a8fa42dd80fe4190ffaf45b2', 'request_id': 'req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.105026 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] The token cache is being invalidate because project eddbc5d2a8fa42dd80fe4190ffaf45b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:45.105323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.106023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e2c3a35e-be75-42d2-92ab-a2c439ae76fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.107578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1235/2484] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:44 2022] DELETE /identity/v3/projects/eddbc5d2a8fa42dd80fe4190ffaf45b2 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:45.108911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3f6f4-a04a-403d-a1a4-b416148de6be None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:45.109254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3f6f4-a04a-403d-a1a4-b416148de6be None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:45.109537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3f6f4-a04a-403d-a1a4-b416148de6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:45.112791 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f02d8c-d4d6-4849-ab64-e452587752e3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:45.119135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:45.119885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:45.120265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:45.120622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:45.123106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:45.130127 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-adc3f6f4-a04a-403d-a1a4-b416148de6be None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:45.131363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-adc3f6f4-a04a-403d-a1a4-b416148de6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:45.137933 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:45.141303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:45.151785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1250/2485] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:45.152697 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:45.154275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:45.199384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1236/2486] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:45.211256 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3f6f4-a04a-403d-a1a4-b416148de6be None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:45.212899 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-adc3f6f4-a04a-403d-a1a4-b416148de6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:45.235326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d199add-021c-4ebd-80d8-7d4f4929146f service placement] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:45.241156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:45.241815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:45.242094 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:45.242954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] PATH_INFO: `/v3/users/b1fd05a3abc54c8ba7edaa5325bb578c` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:45.244182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1fd05a3abc54c8ba7edaa5325bb578c)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:45.246597 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:45.285242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3f6f4-a04a-403d-a1a4-b416148de6be None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:45.289196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c72de-c0c2-40ef-be66-48cc79a4033f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:45.289807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c72de-c0c2-40ef-be66-48cc79a4033f service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:45.290121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c72de-c0c2-40ef-be66-48cc79a4033f service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:45.290394 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c72de-c0c2-40ef-be66-48cc79a4033f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:45.300000 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c72de-c0c2-40ef-be66-48cc79a4033f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:45.301476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c72de-c0c2-40ef-be66-48cc79a4033f service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:45.313425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1237/2487] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:45.542822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1fd05a3abc54c8ba7edaa5325bb578c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.543568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1fd05a3abc54c8ba7edaa5325bb578c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.555995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1fd05a3abc54c8ba7edaa5325bb578c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.700909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1fd05a3abc54c8ba7edaa5325bb578c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.703876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:45.710723 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-e9ddb82e-9473-4788-96be-faa4bc25cf34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1fd05a3abc54c8ba7edaa5325bb578c'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:45.719343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1251/2488] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:45 2022] DELETE /identity/v3/users/b1fd05a3abc54c8ba7edaa5325bb578c => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:46.328240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c72de-c0c2-40ef-be66-48cc79a4033f service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:46.339335 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c72de-c0c2-40ef-be66-48cc79a4033f service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:46.387909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:46.410580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:46.411171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:46.411516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:46.411877 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] PATH_INFO: `/v3/projects/5cd75844078d45da8e68e90d53e70ab0` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:46.413121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd75844078d45da8e68e90d53e70ab0)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:46.415166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:46.558987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd75844078d45da8e68e90d53e70ab0', 'request_id': 'req-d031606a-ab59-4330-90cd-a9f71486de39'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:46.580857 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd75844078d45da8e68e90d53e70ab0', 'request_id': 'req-d031606a-ab59-4330-90cd-a9f71486de39'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:46.639678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd75844078d45da8e68e90d53e70ab0', 'request_id': 'req-d031606a-ab59-4330-90cd-a9f71486de39'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:46.651722 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] The token cache is being invalidate because project 5cd75844078d45da8e68e90d53e70ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:46.657788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:46.657788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d031606a-ab59-4330-90cd-a9f71486de39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:46.657788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1238/2489] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:46 2022] DELETE /identity/v3/projects/5cd75844078d45da8e68e90d53e70ab0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:46.659487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f343273-3909-440d-934c-69ff16bb7e65 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:46.659961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f343273-3909-440d-934c-69ff16bb7e65 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:46.660073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f343273-3909-440d-934c-69ff16bb7e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:46.683171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7f343273-3909-440d-934c-69ff16bb7e65 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:46.684363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7f343273-3909-440d-934c-69ff16bb7e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:46.744483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1252/2490] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:04:46.752660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15ac9a-b024-4b94-b65e-01b0afd6f913 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:46.752945 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15ac9a-b024-4b94-b65e-01b0afd6f913 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:46.753271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15ac9a-b024-4b94-b65e-01b0afd6f913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:46.773090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-1f15ac9a-b024-4b94-b65e-01b0afd6f913 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:46.774357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1f15ac9a-b024-4b94-b65e-01b0afd6f913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:46.783566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1239/2491] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:46.799290 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f343273-3909-440d-934c-69ff16bb7e65 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:46.813936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f343273-3909-440d-934c-69ff16bb7e65 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:46.820091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:46.821547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:46.822040 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:46.822482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:46.823399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:04:46.823874 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:46.826006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:46.840377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1253/2492] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:04:46.845266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15ac9a-b024-4b94-b65e-01b0afd6f913 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:46.855036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15ac9a-b024-4b94-b65e-01b0afd6f913 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:46.861347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb2625-50ea-46b6-a7ec-d721fe329d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:46.862402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb2625-50ea-46b6-a7ec-d721fe329d3b admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:46.862402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb2625-50ea-46b6-a7ec-d721fe329d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:46.862612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb2625-50ea-46b6-a7ec-d721fe329d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:46.863228 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb2625-50ea-46b6-a7ec-d721fe329d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:46.865227 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb2625-50ea-46b6-a7ec-d721fe329d3b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:47.071088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1240/2493] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:46 2022] POST /identity/v3/projects => generated 367 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:47.075192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:47.090574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8fecb-9861-4b25-87f9-39279c769bc4 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:47.169882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4f727-95da-4e61-95f0-a5d396db3224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:47.170773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4f727-95da-4e61-95f0-a5d396db3224 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:47.171230 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4f727-95da-4e61-95f0-a5d396db3224 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:47.171637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4f727-95da-4e61-95f0-a5d396db3224 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:47.172770 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4f727-95da-4e61-95f0-a5d396db3224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:47.176704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4f727-95da-4e61-95f0-a5d396db3224 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:47.482278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1254/2494] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:47 2022] POST /identity/v3/users => generated 284 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:04:47.482453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb2625-50ea-46b6-a7ec-d721fe329d3b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:47.495537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb2625-50ea-46b6-a7ec-d721fe329d3b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:47.502232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7337a2-538f-4f33-9a0e-252f4467aa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:47.502855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7337a2-538f-4f33-9a0e-252f4467aa40 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:47.503306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7337a2-538f-4f33-9a0e-252f4467aa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:47.503442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7337a2-538f-4f33-9a0e-252f4467aa40 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:47.504091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7337a2-538f-4f33-9a0e-252f4467aa40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:47.505779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7337a2-538f-4f33-9a0e-252f4467aa40 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:47.520989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1241/2495] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:47.521902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4f727-95da-4e61-95f0-a5d396db3224 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:47.533265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4f727-95da-4e61-95f0-a5d396db3224 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:47.538726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d4071-7d5a-417a-a2fc-0ca0028b6886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:47.539381 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d4071-7d5a-417a-a2fc-0ca0028b6886 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:47.539698 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d4071-7d5a-417a-a2fc-0ca0028b6886 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:47.539918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d4071-7d5a-417a-a2fc-0ca0028b6886 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:47.540566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d4071-7d5a-417a-a2fc-0ca0028b6886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:47.542375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d4071-7d5a-417a-a2fc-0ca0028b6886 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:47.555214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1255/2496] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:04:47.559507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7337a2-538f-4f33-9a0e-252f4467aa40 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:47.589900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7337a2-538f-4f33-9a0e-252f4467aa40 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:47.596166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378c99f-8cc3-4410-9604-c674f427ceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:47.596915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c99f-8cc3-4410-9604-c674f427ceaf admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:47.597197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c99f-8cc3-4410-9604-c674f427ceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:47.597631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378c99f-8cc3-4410-9604-c674f427ceaf admin admin] PATH_INFO: `/v3/projects/e9fac84a17fc423e83f74dfa242cc474/users/7bc09d5130a7467b83206f481ec1d1e7/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:47.646901 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378c99f-8cc3-4410-9604-c674f427ceaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fac84a17fc423e83f74dfa242cc474, user_id=7bc09d5130a7467b83206f481ec1d1e7, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:47.657587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378c99f-8cc3-4410-9604-c674f427ceaf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:47.811389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1242/2497] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:47 2022] PUT /identity/v3/projects/e9fac84a17fc423e83f74dfa242cc474/users/7bc09d5130a7467b83206f481ec1d1e7/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:47.822485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdbd77-fd46-49e7-8430-bcdd5f29b0e3 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:47.823046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdbd77-fd46-49e7-8430-bcdd5f29b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:47.823046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdbd77-fd46-49e7-8430-bcdd5f29b0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:47.852330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4afdbd77-fd46-49e7-8430-bcdd5f29b0e3 None None] MFA Rules not processed for user `7bc09d5130a7467b83206f481ec1d1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:47.853478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4afdbd77-fd46-49e7-8430-bcdd5f29b0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:47.916507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:47.939596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1256/2498] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:47.944408 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378c99f-8cc3-4410-9604-c674f427ceaf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:48.021375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378c99f-8cc3-4410-9604-c674f427ceaf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:48.033570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eca07-c4c7-4f12-ab6d-0aaf7857f235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:48.034687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eca07-c4c7-4f12-ab6d-0aaf7857f235 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:48.035073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eca07-c4c7-4f12-ab6d-0aaf7857f235 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:48.035532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eca07-c4c7-4f12-ab6d-0aaf7857f235 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:48.036495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eca07-c4c7-4f12-ab6d-0aaf7857f235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:48.040424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eca07-c4c7-4f12-ab6d-0aaf7857f235 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:48.157660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1243/2499] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:47 2022] POST /identity/v3/projects => generated 369 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:48.161405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdbd77-fd46-49e7-8430-bcdd5f29b0e3 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:48.173098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdbd77-fd46-49e7-8430-bcdd5f29b0e3 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:48.188596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:48.276313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe44e3-ef01-4295-9739-b99015f8657f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:48.276993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe44e3-ef01-4295-9739-b99015f8657f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:48.277274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe44e3-ef01-4295-9739-b99015f8657f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:48.277547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe44e3-ef01-4295-9739-b99015f8657f admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:48.278271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe44e3-ef01-4295-9739-b99015f8657f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:48.280822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe44e3-ef01-4295-9739-b99015f8657f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:48.469743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1257/2500] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:48 2022] POST /identity/v3/users => generated 291 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:48.471357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eca07-c4c7-4f12-ab6d-0aaf7857f235 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:48.485887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eca07-c4c7-4f12-ab6d-0aaf7857f235 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:48.492068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176220-cd04-45c1-acc8-423dc0da3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:48.493332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176220-cd04-45c1-acc8-423dc0da3bda admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:48.493332 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176220-cd04-45c1-acc8-423dc0da3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:48.493553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176220-cd04-45c1-acc8-423dc0da3bda admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:48.494179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90176220-cd04-45c1-acc8-423dc0da3bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:48.495856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90176220-cd04-45c1-acc8-423dc0da3bda admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:48.508129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1244/2501] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:48.515423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe44e3-ef01-4295-9739-b99015f8657f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:48.537912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe44e3-ef01-4295-9739-b99015f8657f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:48.541968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b957cf2-313d-4c0e-a45a-d9995307698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:48.542647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b957cf2-313d-4c0e-a45a-d9995307698b admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:48.542948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b957cf2-313d-4c0e-a45a-d9995307698b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:48.543229 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b957cf2-313d-4c0e-a45a-d9995307698b admin admin] PATH_INFO: `/v3/projects/c874be8ccacc43198afb9bd3b3a61540/users/5eae9b25e63c40f594e2cd629125c7b6/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:48.562648 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b957cf2-313d-4c0e-a45a-d9995307698b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c874be8ccacc43198afb9bd3b3a61540, user_id=5eae9b25e63c40f594e2cd629125c7b6, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:48.565499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b957cf2-313d-4c0e-a45a-d9995307698b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:48.723843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1258/2502] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:48 2022] PUT /identity/v3/projects/c874be8ccacc43198afb9bd3b3a61540/users/5eae9b25e63c40f594e2cd629125c7b6/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:48.727820 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176220-cd04-45c1-acc8-423dc0da3bda admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:48.756994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176220-cd04-45c1-acc8-423dc0da3bda admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:48.829262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad876a7-8cea-4baa-8a23-de3f2778a4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:48.829936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad876a7-8cea-4baa-8a23-de3f2778a4db admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:48.830209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad876a7-8cea-4baa-8a23-de3f2778a4db admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:48.830500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad876a7-8cea-4baa-8a23-de3f2778a4db admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:48.831184 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad876a7-8cea-4baa-8a23-de3f2778a4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:48.832780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad876a7-8cea-4baa-8a23-de3f2778a4db admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:48.844120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1245/2503] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:48.848450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b957cf2-313d-4c0e-a45a-d9995307698b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:48.861077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b957cf2-313d-4c0e-a45a-d9995307698b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:48.871294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:48.872351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:48.872991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:48.874050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] PATH_INFO: `/v3/projects/c874be8ccacc43198afb9bd3b3a61540/users/5eae9b25e63c40f594e2cd629125c7b6/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:48.876406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c874be8ccacc43198afb9bd3b3a61540, user_id=5eae9b25e63c40f594e2cd629125c7b6, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:48.884763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:48.895552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1259/2504] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:48 2022] PUT /identity/v3/projects/c874be8ccacc43198afb9bd3b3a61540/users/5eae9b25e63c40f594e2cd629125c7b6/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:48.908766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa193-7076-40ba-962b-df0ddb9ac1b2 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:48.909200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa193-7076-40ba-962b-df0ddb9ac1b2 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:48.909551 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa193-7076-40ba-962b-df0ddb9ac1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:48.948388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-580aa193-7076-40ba-962b-df0ddb9ac1b2 None None] MFA Rules not processed for user `5eae9b25e63c40f594e2cd629125c7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:48.949623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-580aa193-7076-40ba-962b-df0ddb9ac1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:49.024062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1246/2505] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:49.034844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:49.036640 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:49.115526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f88f7-72bd-4a63-a48e-cd65789c1428 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:49.123669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22801a-4358-4dc9-8b33-82ebe2ed5ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:49.123669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22801a-4358-4dc9-8b33-82ebe2ed5ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:49.123669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22801a-4358-4dc9-8b33-82ebe2ed5ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:49.123669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22801a-4358-4dc9-8b33-82ebe2ed5ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:49.133279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22801a-4358-4dc9-8b33-82ebe2ed5ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:49.135641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22801a-4358-4dc9-8b33-82ebe2ed5ba2 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:49.152401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1260/2506] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:50.525621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dfcd2-b6a3-4ea4-9de9-ce53c228c611 None None] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:50.526171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dfcd2-b6a3-4ea4-9de9-ce53c228c611 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:50.526171 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dfcd2-b6a3-4ea4-9de9-ce53c228c611 None None] PATH_INFO: `/` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:50.527516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1247/2507] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 16:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 16:04:50.544211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3a5a-bf1c-41d1-938e-ecee7c265e18 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:50.547530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3a5a-bf1c-41d1-938e-ecee7c265e18 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:50.550744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3a5a-bf1c-41d1-938e-ecee7c265e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:50.635780 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3fed3a5a-bf1c-41d1-938e-ecee7c265e18 None None] MFA Rules not processed for user `b7ad758153334ec28940156d09495ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:50.637185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3fed3a5a-bf1c-41d1-938e-ecee7c265e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:50.670026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:50.693311 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1261/2508] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 16:04:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:50.707323 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dfcd2-b6a3-4ea4-9de9-ce53c228c611 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:50.709254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-124dfcd2-b6a3-4ea4-9de9-ce53c228c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:50.722971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dfcd2-b6a3-4ea4-9de9-ce53c228c611 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:50.726663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603c314-13f2-4b64-9f08-120b2b7542f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:50.727299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c314-13f2-4b64-9f08-120b2b7542f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:50.727570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c314-13f2-4b64-9f08-120b2b7542f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:50.728024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603c314-13f2-4b64-9f08-120b2b7542f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:50.735344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603c314-13f2-4b64-9f08-120b2b7542f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:50.736823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603c314-13f2-4b64-9f08-120b2b7542f2 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:50.761484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1248/2509] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:50.931933 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f954b-04dc-40dd-8a68-48d9cfed4de6 None None] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:50.932322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f954b-04dc-40dd-8a68-48d9cfed4de6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:50.932815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f954b-04dc-40dd-8a68-48d9cfed4de6 None None] PATH_INFO: `/` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:50.935070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1262/2510] 10.222.0.94 () {54 vars in 921 bytes} [Fri Aug 26 16:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 16:04:50.941462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d672a69-80b4-4817-b424-83de821c0338 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:50.941748 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d672a69-80b4-4817-b424-83de821c0338 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:50.942027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d672a69-80b4-4817-b424-83de821c0338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:50.982351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-8d672a69-80b4-4817-b424-83de821c0338 None None] MFA Rules not processed for user `500d2f2bd83d499aa2814c6d7cc71795`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:50.984757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8d672a69-80b4-4817-b424-83de821c0338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:51.075164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1249/2511] 10.222.0.94 () {58 vars in 1018 bytes} [Fri Aug 26 16:04:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:51.088162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f954b-04dc-40dd-8a68-48d9cfed4de6 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:51.088162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b25f954b-04dc-40dd-8a68-48d9cfed4de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:51.101447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f954b-04dc-40dd-8a68-48d9cfed4de6 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:51.104922 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099029-eb50-4ae9-a84b-b09660d2f8d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:51.105604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099029-eb50-4ae9-a84b-b09660d2f8d2 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:51.105872 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099029-eb50-4ae9-a84b-b09660d2f8d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:51.106136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099029-eb50-4ae9-a84b-b09660d2f8d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:51.117985 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7099029-eb50-4ae9-a84b-b09660d2f8d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:51.119431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7099029-eb50-4ae9-a84b-b09660d2f8d2 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:51.132059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1263/2512] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:51.327209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:52.042429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d672a69-80b4-4817-b424-83de821c0338 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:52.055062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d672a69-80b4-4817-b424-83de821c0338 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:52.062120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:52.063689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:52.069458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:52.069458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:52.069458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:52.150558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:52.152021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:52.165696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1250/2513] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:52.773496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6a9a-9cae-41ff-a4a1-98d16f7d7bfa None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:52.774555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6a9a-9cae-41ff-a4a1-98d16f7d7bfa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:52.775258 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f6a9a-9cae-41ff-a4a1-98d16f7d7bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:52.807614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-019f6a9a-9cae-41ff-a4a1-98d16f7d7bfa None None] MFA Rules not processed for user `c911a9324cb744a6a1aeb068565d3f89`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:52.809002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-019f6a9a-9cae-41ff-a4a1-98d16f7d7bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:52.901922 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1264/2514] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:52.905597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:52.909900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:52.922638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a978d-5ee8-4ec2-a79d-82581b5c4d4e service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:52.933259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae090e-0bd0-4fbc-8c14-dc36e299eea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:52.934174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae090e-0bd0-4fbc-8c14-dc36e299eea5 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:52.934769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae090e-0bd0-4fbc-8c14-dc36e299eea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:52.935424 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae090e-0bd0-4fbc-8c14-dc36e299eea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:52.936962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae090e-0bd0-4fbc-8c14-dc36e299eea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:52.939134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae090e-0bd0-4fbc-8c14-dc36e299eea5 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:53.056097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1251/2515] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:52 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:53.059630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f6a9a-9cae-41ff-a4a1-98d16f7d7bfa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:53.072715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f6a9a-9cae-41ff-a4a1-98d16f7d7bfa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:53.150156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1a99-ddce-41f6-bf99-6f332f0d7845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:53.150773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1a99-ddce-41f6-bf99-6f332f0d7845 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:53.151086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1a99-ddce-41f6-bf99-6f332f0d7845 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:53.151354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1a99-ddce-41f6-bf99-6f332f0d7845 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:53.151983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e1a99-ddce-41f6-bf99-6f332f0d7845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:53.155492 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e1a99-ddce-41f6-bf99-6f332f0d7845 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:53.273959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1265/2516] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:53 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:53.280126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae090e-0bd0-4fbc-8c14-dc36e299eea5 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:53.295091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae090e-0bd0-4fbc-8c14-dc36e299eea5 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:53.300641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4fed1-e75a-4e63-8587-5f0f3e357aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:53.301418 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4fed1-e75a-4e63-8587-5f0f3e357aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:53.301789 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4fed1-e75a-4e63-8587-5f0f3e357aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:53.302185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4fed1-e75a-4e63-8587-5f0f3e357aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:53.303114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4fed1-e75a-4e63-8587-5f0f3e357aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:53.304714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4fed1-e75a-4e63-8587-5f0f3e357aa1 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:53.316566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1252/2517] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:53.322256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1a99-ddce-41f6-bf99-6f332f0d7845 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:53.342750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1a99-ddce-41f6-bf99-6f332f0d7845 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:53.348047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5108d4-ea12-4948-8855-316fefb3d8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:53.348760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5108d4-ea12-4948-8855-316fefb3d8db admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:53.349036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5108d4-ea12-4948-8855-316fefb3d8db admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:53.349310 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5108d4-ea12-4948-8855-316fefb3d8db admin admin] PATH_INFO: `/v3/projects/d89e0264a19e4a79a8fe884c40a4f593/users/81f2329e50b347b89b6f9a980bea2f07/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:53.373553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5108d4-ea12-4948-8855-316fefb3d8db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d89e0264a19e4a79a8fe884c40a4f593, user_id=81f2329e50b347b89b6f9a980bea2f07, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:53.377095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5108d4-ea12-4948-8855-316fefb3d8db admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:53.396032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4fed1-e75a-4e63-8587-5f0f3e357aa1 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:53.411614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4fed1-e75a-4e63-8587-5f0f3e357aa1 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:53.416580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:53.417286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:53.417586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:53.417858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] PATH_INFO: `/v3/users/7bc09d5130a7467b83206f481ec1d1e7` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:53.419103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bc09d5130a7467b83206f481ec1d1e7)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:53.421569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:53.469850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1266/2518] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:53 2022] PUT /identity/v3/projects/d89e0264a19e4a79a8fe884c40a4f593/users/81f2329e50b347b89b6f9a980bea2f07/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:53.473513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5108d4-ea12-4948-8855-316fefb3d8db admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:53.504055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:53.584295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5108d4-ea12-4948-8855-316fefb3d8db admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:53.594080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dcc23-2751-49eb-9ad1-cd1c999868ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:53.594995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dcc23-2751-49eb-9ad1-cd1c999868ba admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:53.595449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dcc23-2751-49eb-9ad1-cd1c999868ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:53.596339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dcc23-2751-49eb-9ad1-cd1c999868ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:53.598225 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dcc23-2751-49eb-9ad1-cd1c999868ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:53.600199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dcc23-2751-49eb-9ad1-cd1c999868ba admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:53.614577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1267/2519] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:04:53.620032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dcc23-2751-49eb-9ad1-cd1c999868ba admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:53.634414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dcc23-2751-49eb-9ad1-cd1c999868ba admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:53.640451 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b4bf5-9916-4abc-b774-68755e7549f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:53.641280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b4bf5-9916-4abc-b774-68755e7549f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:53.641850 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b4bf5-9916-4abc-b774-68755e7549f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:53.642136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b4bf5-9916-4abc-b774-68755e7549f5 admin admin] PATH_INFO: `/v3/projects/d89e0264a19e4a79a8fe884c40a4f593/users/81f2329e50b347b89b6f9a980bea2f07/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:53.644661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b4bf5-9916-4abc-b774-68755e7549f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d89e0264a19e4a79a8fe884c40a4f593, user_id=81f2329e50b347b89b6f9a980bea2f07, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:53.647626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b4bf5-9916-4abc-b774-68755e7549f5 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:53.654564 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bc09d5130a7467b83206f481ec1d1e7'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:53.655802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bc09d5130a7467b83206f481ec1d1e7'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:53.669083 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1268/2520] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:53 2022] PUT /identity/v3/projects/d89e0264a19e4a79a8fe884c40a4f593/users/81f2329e50b347b89b6f9a980bea2f07/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:53.695460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bc09d5130a7467b83206f481ec1d1e7'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:53.696316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bc09d5130a7467b83206f481ec1d1e7'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:53.700577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:53.835229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bc09d5130a7467b83206f481ec1d1e7'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:53.853581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1253/2521] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:53 2022] DELETE /identity/v3/users/7bc09d5130a7467b83206f481ec1d1e7 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:53.855242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d878126-421f-461f-9c24-258483561b6d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:53.855642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d878126-421f-461f-9c24-258483561b6d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:53.855776 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d878126-421f-461f-9c24-258483561b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:53.897486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7d878126-421f-461f-9c24-258483561b6d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:53.899470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7d878126-421f-461f-9c24-258483561b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:53.991089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1269/2522] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:54.008203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:54.014412 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:54.159202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc849038-d27f-4b38-89c7-94c034df38b2 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:54.165417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44fa63-f271-4f79-9c04-4d6fe2aaae32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:54.166732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44fa63-f271-4f79-9c04-4d6fe2aaae32 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:54.167208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44fa63-f271-4f79-9c04-4d6fe2aaae32 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:54.168296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44fa63-f271-4f79-9c04-4d6fe2aaae32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:54.185118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44fa63-f271-4f79-9c04-4d6fe2aaae32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:54.186895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44fa63-f271-4f79-9c04-4d6fe2aaae32 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:54.202403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1254/2523] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:04:54.884849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d878126-421f-461f-9c24-258483561b6d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:54.887029 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7d878126-421f-461f-9c24-258483561b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:54.909322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d878126-421f-461f-9c24-258483561b6d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:54.918711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:54.919658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:54.920120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:54.920839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] PATH_INFO: `/v3/projects/e9fac84a17fc423e83f74dfa242cc474` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:54.923534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9fac84a17fc423e83f74dfa242cc474)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:54.925958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:55.108028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9fac84a17fc423e83f74dfa242cc474', 'request_id': 'req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.113763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9fac84a17fc423e83f74dfa242cc474', 'request_id': 'req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.239548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9fac84a17fc423e83f74dfa242cc474', 'request_id': 'req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.261779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] The token cache is being invalidate because project e9fac84a17fc423e83f74dfa242cc474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:55.262266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.263057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.265918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1270/2524] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:54 2022] DELETE /identity/v3/projects/e9fac84a17fc423e83f74dfa242cc474 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:55.269759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44fa63-f271-4f79-9c04-4d6fe2aaae32 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:55.281706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44fa63-f271-4f79-9c04-4d6fe2aaae32 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:55.351328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:55.352398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:55.352900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:55.353378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] PATH_INFO: `/v3/users/5eae9b25e63c40f594e2cd629125c7b6` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:55.354792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eae9b25e63c40f594e2cd629125c7b6)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:55.357934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:55.536907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eae9b25e63c40f594e2cd629125c7b6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.539190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eae9b25e63c40f594e2cd629125c7b6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.554587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eae9b25e63c40f594e2cd629125c7b6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.555379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eae9b25e63c40f594e2cd629125c7b6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.668739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-837bfce3-4fe6-4ba8-9948-d3c2a20352e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eae9b25e63c40f594e2cd629125c7b6'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:55.700794 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1255/2525] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:55 2022] DELETE /identity/v3/users/5eae9b25e63c40f594e2cd629125c7b6 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:56.239090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:56.830395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:56.831871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:56.849696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e9fd5-3983-45b2-8ad6-b74a8aa1f581 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:56.958063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:57.023118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:57.023885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:57.024370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:57.024797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] PATH_INFO: `/v3/projects/c874be8ccacc43198afb9bd3b3a61540` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:57.026432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c874be8ccacc43198afb9bd3b3a61540)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:57.028582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:57.226328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c874be8ccacc43198afb9bd3b3a61540', 'request_id': 'req-622dd389-67f3-4b02-9dd5-cb3123ad38c5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:57.284150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c874be8ccacc43198afb9bd3b3a61540', 'request_id': 'req-622dd389-67f3-4b02-9dd5-cb3123ad38c5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:57.398246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c874be8ccacc43198afb9bd3b3a61540', 'request_id': 'req-622dd389-67f3-4b02-9dd5-cb3123ad38c5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:57.444412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] The token cache is being invalidate because project c874be8ccacc43198afb9bd3b3a61540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:04:57.444879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:57.448450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-622dd389-67f3-4b02-9dd5-cb3123ad38c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:57.461877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1271/2526] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:56 2022] DELETE /identity/v3/projects/c874be8ccacc43198afb9bd3b3a61540 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:57.463768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b286-60b4-4919-b67b-148cdabe77d0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:57.464428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b286-60b4-4919-b67b-148cdabe77d0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:57.464428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b286-60b4-4919-b67b-148cdabe77d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:57.504289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5207b286-60b4-4919-b67b-148cdabe77d0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:57.505628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5207b286-60b4-4919-b67b-148cdabe77d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:57.633236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1256/2527] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:57.641828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5dff2-8496-4b8e-9fc9-0ced6ef991cf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:57.642352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5dff2-8496-4b8e-9fc9-0ced6ef991cf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:57.642797 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5dff2-8496-4b8e-9fc9-0ced6ef991cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:57.686183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d5d5dff2-8496-4b8e-9fc9-0ced6ef991cf None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:57.687898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d5d5dff2-8496-4b8e-9fc9-0ced6ef991cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:57.698601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1272/2528] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:57.703236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207b286-60b4-4919-b67b-148cdabe77d0 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:57.715024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207b286-60b4-4919-b67b-148cdabe77d0 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:57.729414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:57.730328 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:57.730764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:57.731360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:57.732900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:04:57.733528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:57.735814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:57.772680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1257/2529] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:04:57.776012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5dff2-8496-4b8e-9fc9-0ced6ef991cf None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:57.788488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5dff2-8496-4b8e-9fc9-0ced6ef991cf None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:57.802522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c63ac6-a46d-4d5a-84df-f23b54a7cbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:57.803220 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c63ac6-a46d-4d5a-84df-f23b54a7cbdf admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:57.803654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c63ac6-a46d-4d5a-84df-f23b54a7cbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:57.804071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c63ac6-a46d-4d5a-84df-f23b54a7cbdf admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:57.804898 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c63ac6-a46d-4d5a-84df-f23b54a7cbdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:57.807630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c63ac6-a46d-4d5a-84df-f23b54a7cbdf admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:58.051843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1273/2530] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:57 2022] POST /identity/v3/projects => generated 377 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:58.053779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:58.069379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46c6e2-4ba7-4830-a3ca-13225c300ac2 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:58.161103 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc52995-c9d5-40e7-9843-5aafd01a75a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:58.161832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc52995-c9d5-40e7-9843-5aafd01a75a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:58.162181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc52995-c9d5-40e7-9843-5aafd01a75a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:58.162668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc52995-c9d5-40e7-9843-5aafd01a75a7 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:58.163476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc52995-c9d5-40e7-9843-5aafd01a75a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:58.169198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc52995-c9d5-40e7-9843-5aafd01a75a7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:58.328877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c63ac6-a46d-4d5a-84df-f23b54a7cbdf admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:58.334807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1258/2531] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:58 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:04:58.341091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c63ac6-a46d-4d5a-84df-f23b54a7cbdf admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:58.347327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c1ad6-6fc2-46a1-b0d6-e16d863dfc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:58.348216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c1ad6-6fc2-46a1-b0d6-e16d863dfc81 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:58.348645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c1ad6-6fc2-46a1-b0d6-e16d863dfc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:58.349071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c1ad6-6fc2-46a1-b0d6-e16d863dfc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:58.349873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c1ad6-6fc2-46a1-b0d6-e16d863dfc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:58.351501 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c1ad6-6fc2-46a1-b0d6-e16d863dfc81 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:58.364836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1274/2532] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:04:58.368821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc52995-c9d5-40e7-9843-5aafd01a75a7 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:58.380351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc52995-c9d5-40e7-9843-5aafd01a75a7 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:58.386311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:58.386944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:58.387659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:58.387659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:58.388152 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:58.389993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:58.403200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1259/2533] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:58.407437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c1ad6-6fc2-46a1-b0d6-e16d863dfc81 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:58.420546 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c1ad6-6fc2-46a1-b0d6-e16d863dfc81 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:58.425909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cb4f5-5831-431f-889b-8fc96c1a01e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:58.426852 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb4f5-5831-431f-889b-8fc96c1a01e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:58.427410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb4f5-5831-431f-889b-8fc96c1a01e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:58.427849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb4f5-5831-431f-889b-8fc96c1a01e0 admin admin] PATH_INFO: `/v3/projects/6f052a3543484b519aabe91832d6568f/users/f074952b3564456abe94dcd0be50531d/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:58.448500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cb4f5-5831-431f-889b-8fc96c1a01e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f052a3543484b519aabe91832d6568f, user_id=f074952b3564456abe94dcd0be50531d, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:58.451589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cb4f5-5831-431f-889b-8fc96c1a01e0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:58.488316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:58.488316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:58.502279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc1921-6e32-48e6-a150-72f8dc48852a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:58.507427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:58.508072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:58.508343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:58.508790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] PATH_INFO: `/v3/users/b50041d36b0b430badd6442a2f09f063` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:58.509941 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b50041d36b0b430badd6442a2f09f063)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:58.512023 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:58.617523 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1275/2534] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:58 2022] PUT /identity/v3/projects/6f052a3543484b519aabe91832d6568f/users/f074952b3564456abe94dcd0be50531d/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:58.620070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ce997-97c1-4875-9f0c-689db58e6347 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:58.620337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ce997-97c1-4875-9f0c-689db58e6347 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:58.620603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ce997-97c1-4875-9f0c-689db58e6347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:58.662446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-898ce997-97c1-4875-9f0c-689db58e6347 None None] MFA Rules not processed for user `f074952b3564456abe94dcd0be50531d`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:58.664077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-898ce997-97c1-4875-9f0c-689db58e6347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:58.717138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b50041d36b0b430badd6442a2f09f063'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:58.717138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b50041d36b0b430badd6442a2f09f063'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:58.729669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b50041d36b0b430badd6442a2f09f063'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:58.730750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b50041d36b0b430badd6442a2f09f063'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:58.752414 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1276/2535] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:04:58.757465 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ce997-97c1-4875-9f0c-689db58e6347 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:58.758967 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-898ce997-97c1-4875-9f0c-689db58e6347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:58.817819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b50041d36b0b430badd6442a2f09f063'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:58.847700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ce997-97c1-4875-9f0c-689db58e6347 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:58.853303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f885681-9d8b-4385-954f-f8e7d0ef84d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:58.854065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f885681-9d8b-4385-954f-f8e7d0ef84d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:58.854437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f885681-9d8b-4385-954f-f8e7d0ef84d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:58.854798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f885681-9d8b-4385-954f-f8e7d0ef84d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:58.855908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f885681-9d8b-4385-954f-f8e7d0ef84d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:58.858782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f885681-9d8b-4385-954f-f8e7d0ef84d2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:58.861669 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1260/2536] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:04:58 2022] DELETE /identity/v3/users/b50041d36b0b430badd6442a2f09f063 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:59.022113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1277/2537] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:04:58 2022] POST /identity/v3/projects => generated 377 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:04:59.023169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:59.112381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b15b2-6e1e-4379-b469-070400d0182f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:59.179267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f3e26-e664-4f7a-9f4d-c6bc9fbbc96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:59.179894 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f3e26-e664-4f7a-9f4d-c6bc9fbbc96c admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:59.180163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f3e26-e664-4f7a-9f4d-c6bc9fbbc96c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:59.180751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f3e26-e664-4f7a-9f4d-c6bc9fbbc96c admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:59.181196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f3e26-e664-4f7a-9f4d-c6bc9fbbc96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:59.183987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f3e26-e664-4f7a-9f4d-c6bc9fbbc96c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:59.190417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:59.309185 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1261/2538] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:04:59 2022] POST /identity/v3/users => generated 295 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:04:59.314021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f885681-9d8b-4385-954f-f8e7d0ef84d2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:59.325674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f885681-9d8b-4385-954f-f8e7d0ef84d2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:59.331600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50fe31-4de4-4428-b695-13b220a3b272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:59.332399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50fe31-4de4-4428-b695-13b220a3b272 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:59.332798 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50fe31-4de4-4428-b695-13b220a3b272 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:59.333170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50fe31-4de4-4428-b695-13b220a3b272 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:59.333896 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50fe31-4de4-4428-b695-13b220a3b272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:59.335683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50fe31-4de4-4428-b695-13b220a3b272 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:59.351868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1278/2539] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:04:59.353173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f3e26-e664-4f7a-9f4d-c6bc9fbbc96c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:59.366599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f3e26-e664-4f7a-9f4d-c6bc9fbbc96c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:59.371213 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:59.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:59.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:59.376399 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] PATH_INFO: `/v3/projects/15c3bb3a8acc4897bbf023e5d5efb496/users/d7b91e6082c14e899497277c27c25bfe/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:59.393069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c3bb3a8acc4897bbf023e5d5efb496, user_id=d7b91e6082c14e899497277c27c25bfe, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:59.397257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:59.476690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1262/2540] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:59 2022] PUT /identity/v3/projects/15c3bb3a8acc4897bbf023e5d5efb496/users/d7b91e6082c14e899497277c27c25bfe/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:04:59.480753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50fe31-4de4-4428-b695-13b220a3b272 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:59.492121 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50fe31-4de4-4428-b695-13b220a3b272 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:59.561496 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e5a1-bd1a-4e84-90ca-6c13df1beae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:59.563319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e5a1-bd1a-4e84-90ca-6c13df1beae0 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:59.563646 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e5a1-bd1a-4e84-90ca-6c13df1beae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:59.564008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e5a1-bd1a-4e84-90ca-6c13df1beae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:59.564761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12e5a1-bd1a-4e84-90ca-6c13df1beae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:59.566981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12e5a1-bd1a-4e84-90ca-6c13df1beae0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:59.571279 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:59.572032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:04:59.577371 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:59.598356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1279/2541] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:04:59.602612 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e5a1-bd1a-4e84-90ca-6c13df1beae0 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:59.610626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf00ed-708e-4b88-a173-89730b43d5d2 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:59.618197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:59.619273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:59.619824 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:59.621106 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] PATH_INFO: `/v3/projects/d95f731096d74a089f4161f5db69280c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:59.622856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95f731096d74a089f4161f5db69280c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:59.625719 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:59.627356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e5a1-bd1a-4e84-90ca-6c13df1beae0 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:59.637884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fb93f-fcdf-45bf-ae4a-f6906305cfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:04:59.646093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fb93f-fcdf-45bf-ae4a-f6906305cfe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:59.646683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fb93f-fcdf-45bf-ae4a-f6906305cfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:59.647228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fb93f-fcdf-45bf-ae4a-f6906305cfe1 admin admin] PATH_INFO: `/v3/projects/15c3bb3a8acc4897bbf023e5d5efb496/users/d7b91e6082c14e899497277c27c25bfe/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:59.649648 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4fb93f-fcdf-45bf-ae4a-f6906305cfe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c3bb3a8acc4897bbf023e5d5efb496, user_id=d7b91e6082c14e899497277c27c25bfe, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:04:59.652677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4fb93f-fcdf-45bf-ae4a-f6906305cfe1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:04:59.679645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1280/2542] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:04:59 2022] PUT /identity/v3/projects/15c3bb3a8acc4897bbf023e5d5efb496/users/d7b91e6082c14e899497277c27c25bfe/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:04:59.692913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50256ca5-c821-4667-b690-708a1980ed33 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:04:59.693005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50256ca5-c821-4667-b690-708a1980ed33 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:04:59.695149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50256ca5-c821-4667-b690-708a1980ed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:04:59.733864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-50256ca5-c821-4667-b690-708a1980ed33 None None] MFA Rules not processed for user `d7b91e6082c14e899497277c27c25bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:04:59.735520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-50256ca5-c821-4667-b690-708a1980ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:59.833064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1281/2543] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:04:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:04:59.849094 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95f731096d74a089f4161f5db69280c', 'request_id': 'req-b6608782-bc51-4bde-8149-b71189cef463'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:59.853427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50256ca5-c821-4667-b690-708a1980ed33 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:04:59.855606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-50256ca5-c821-4667-b690-708a1980ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:04:59.856376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95f731096d74a089f4161f5db69280c', 'request_id': 'req-b6608782-bc51-4bde-8149-b71189cef463'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:04:59.987236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50256ca5-c821-4667-b690-708a1980ed33 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:04:59.993643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95f731096d74a089f4161f5db69280c', 'request_id': 'req-b6608782-bc51-4bde-8149-b71189cef463'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:00.014535 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] The token cache is being invalidate because project d95f731096d74a089f4161f5db69280c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:00.015196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:00.016114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b6608782-bc51-4bde-8149-b71189cef463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:00.021942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1263/2544] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:04:59 2022] DELETE /identity/v3/projects/d95f731096d74a089f4161f5db69280c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:00.023672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b016a8-d667-4b4a-977d-e2b1bd0d5a70 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.024164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b016a8-d667-4b4a-977d-e2b1bd0d5a70 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.024647 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b016a8-d667-4b4a-977d-e2b1bd0d5a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.056142 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-55b016a8-d667-4b4a-977d-e2b1bd0d5a70 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:00.058454 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-55b016a8-d667-4b4a-977d-e2b1bd0d5a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:00.060326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:00.061457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.061820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.062097 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.064098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:00.125779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1264/2545] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:00.133495 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:00.134053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895563f-58d5-40d4-ad24-80511a8cc601 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.134342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895563f-58d5-40d4-ad24-80511a8cc601 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.134619 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895563f-58d5-40d4-ad24-80511a8cc601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.135458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:00.151161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1282/2546] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:04:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:00.157167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-0895563f-58d5-40d4-ad24-80511a8cc601 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:00.158469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0895563f-58d5-40d4-ad24-80511a8cc601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:00.172064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1265/2547] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:00.172902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:00.191486 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fbe17-33bf-4d2b-b124-e3949f870343 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:00.196927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:00.197649 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.197961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.198182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.198899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:00.199234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:00.216686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:00.224079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895563f-58d5-40d4-ad24-80511a8cc601 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:00.231555 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1283/2548] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 16:05:00.237109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895563f-58d5-40d4-ad24-80511a8cc601 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:00.243561 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff354f27-cccb-410d-95a6-29ba0730f0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:00.244646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354f27-cccb-410d-95a6-29ba0730f0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.244968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354f27-cccb-410d-95a6-29ba0730f0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.245471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354f27-cccb-410d-95a6-29ba0730f0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.246732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff354f27-cccb-410d-95a6-29ba0730f0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:00.251565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff354f27-cccb-410d-95a6-29ba0730f0b6 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:00.350449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:00.359860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1266/2549] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:00 2022] POST /identity/v3/projects => generated 403 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:00.362505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c1b84-f9b0-4263-ad58-6d999411cdf0 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:00.434573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47882-62de-4250-86e2-51897dee5db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:00.436010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47882-62de-4250-86e2-51897dee5db4 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.441408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47882-62de-4250-86e2-51897dee5db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.441408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47882-62de-4250-86e2-51897dee5db4 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.441408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47882-62de-4250-86e2-51897dee5db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:00.441408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47882-62de-4250-86e2-51897dee5db4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:00.562437 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1284/2550] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:00 2022] POST /identity/v3/users => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:00.566966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff354f27-cccb-410d-95a6-29ba0730f0b6 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:00.578649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff354f27-cccb-410d-95a6-29ba0730f0b6 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:00.583439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0a30f-ba10-4854-b1e9-e5f852495ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:00.584298 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0a30f-ba10-4854-b1e9-e5f852495ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.584766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0a30f-ba10-4854-b1e9-e5f852495ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.585233 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0a30f-ba10-4854-b1e9-e5f852495ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.586073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0a30f-ba10-4854-b1e9-e5f852495ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:00.587908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0a30f-ba10-4854-b1e9-e5f852495ec5 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:00.604585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47882-62de-4250-86e2-51897dee5db4 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:00.605002 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1267/2551] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:00.616545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47882-62de-4250-86e2-51897dee5db4 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:00.621763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1027b-8ccd-40c7-b443-7985350920be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:00.622559 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1027b-8ccd-40c7-b443-7985350920be admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.623961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1027b-8ccd-40c7-b443-7985350920be admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.624613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1027b-8ccd-40c7-b443-7985350920be admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.625271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1027b-8ccd-40c7-b443-7985350920be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:00.626865 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1027b-8ccd-40c7-b443-7985350920be admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:00.642972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1285/2552] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:00.644041 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0a30f-ba10-4854-b1e9-e5f852495ec5 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:00.654642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0a30f-ba10-4854-b1e9-e5f852495ec5 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:00.661954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43a4f3-081c-4e70-ab0d-8c19831e85df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:00.662630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a4f3-081c-4e70-ab0d-8c19831e85df admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.662907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a4f3-081c-4e70-ab0d-8c19831e85df admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.663207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43a4f3-081c-4e70-ab0d-8c19831e85df admin admin] PATH_INFO: `/v3/projects/bad02b350b0148d2a017740cf79b35c5/users/8b5b9648afdf45859b2df3aba84a589d/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.697874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43a4f3-081c-4e70-ab0d-8c19831e85df admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad02b350b0148d2a017740cf79b35c5, user_id=8b5b9648afdf45859b2df3aba84a589d, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:00.700601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43a4f3-081c-4e70-ab0d-8c19831e85df admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:00.810235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1268/2553] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:00 2022] PUT /identity/v3/projects/bad02b350b0148d2a017740cf79b35c5/users/8b5b9648afdf45859b2df3aba84a589d/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:00.823399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f36b-b2a8-4b17-a414-b59db7da7aaa None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:00.823707 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f36b-b2a8-4b17-a414-b59db7da7aaa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:00.823950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f36b-b2a8-4b17-a414-b59db7da7aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:00.855536 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6c40f36b-b2a8-4b17-a414-b59db7da7aaa None None] MFA Rules not processed for user `8b5b9648afdf45859b2df3aba84a589d`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:00.856838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6c40f36b-b2a8-4b17-a414-b59db7da7aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:01.014462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1286/2554] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:01.017984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43a4f3-081c-4e70-ab0d-8c19831e85df admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:01.174088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43a4f3-081c-4e70-ab0d-8c19831e85df admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:01.179122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb2bb-2463-4679-9b4f-98628f73de4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:01.179758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb2bb-2463-4679-9b4f-98628f73de4d admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.180021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb2bb-2463-4679-9b4f-98628f73de4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.180295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb2bb-2463-4679-9b4f-98628f73de4d admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.180944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb2bb-2463-4679-9b4f-98628f73de4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:01.182902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb2bb-2463-4679-9b4f-98628f73de4d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:01.227787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:01.332200 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1269/2555] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:01 2022] POST /identity/v3/projects => generated 405 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:01.334088 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f36b-b2a8-4b17-a414-b59db7da7aaa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:01.350745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f36b-b2a8-4b17-a414-b59db7da7aaa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:01.385685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7133e-8fdf-4926-828d-5ed5e8b0dc34 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.385968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7133e-8fdf-4926-828d-5ed5e8b0dc34 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.386243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7133e-8fdf-4926-828d-5ed5e8b0dc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.420821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-75a7133e-8fdf-4926-828d-5ed5e8b0dc34 None None] MFA Rules not processed for user `81f2329e50b347b89b6f9a980bea2f07`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:01.422388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-75a7133e-8fdf-4926-828d-5ed5e8b0dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:01.429816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4b383-86e4-48cc-bc71-36f79ad4f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:01.430608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4b383-86e4-48cc-bc71-36f79ad4f74f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.431050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4b383-86e4-48cc-bc71-36f79ad4f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.431420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4b383-86e4-48cc-bc71-36f79ad4f74f admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.432150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4b383-86e4-48cc-bc71-36f79ad4f74f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:01.434758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4b383-86e4-48cc-bc71-36f79ad4f74f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:01.502956 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1270/2556] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:01.523504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7133e-8fdf-4926-828d-5ed5e8b0dc34 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:01.529438 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-75a7133e-8fdf-4926-828d-5ed5e8b0dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:01.538159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1287/2557] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:01 2022] POST /identity/v3/users => generated 309 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:01.543344 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4b383-86e4-48cc-bc71-36f79ad4f74f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:01.561515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4b383-86e4-48cc-bc71-36f79ad4f74f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:01.568195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d1edf-6b73-4743-ba97-afad1ce50c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:01.569132 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1edf-6b73-4743-ba97-afad1ce50c82 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.569600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1edf-6b73-4743-ba97-afad1ce50c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.570678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d1edf-6b73-4743-ba97-afad1ce50c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.571312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d1edf-6b73-4743-ba97-afad1ce50c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:01.573273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d1edf-6b73-4743-ba97-afad1ce50c82 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:01.594215 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1288/2558] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:01.598557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d1edf-6b73-4743-ba97-afad1ce50c82 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:01.611307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d1edf-6b73-4743-ba97-afad1ce50c82 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:01.619933 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91f8be-42a7-4d85-aa21-2dbdeba9963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:01.621658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f8be-42a7-4d85-aa21-2dbdeba9963f admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.622141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f8be-42a7-4d85-aa21-2dbdeba9963f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.622595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f8be-42a7-4d85-aa21-2dbdeba9963f admin admin] PATH_INFO: `/v3/projects/9431661d673f4d3884c1b1bfb9a36856/users/69782c1ca83843cba684ac32e8ede079/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.649545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91f8be-42a7-4d85-aa21-2dbdeba9963f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9431661d673f4d3884c1b1bfb9a36856, user_id=69782c1ca83843cba684ac32e8ede079, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:01.654539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91f8be-42a7-4d85-aa21-2dbdeba9963f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:01.663662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7133e-8fdf-4926-828d-5ed5e8b0dc34 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:01.671178 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:01.672151 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.672649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.673116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.675797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:01.764092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:01.764092 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:01.768052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1271/2559] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:01.808930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1289/2560] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:01 2022] PUT /identity/v3/projects/9431661d673f4d3884c1b1bfb9a36856/users/69782c1ca83843cba684ac32e8ede079/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:01.810003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:01.823007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672bae6-d1c2-4aed-be3d-2cb8271ed34d service nova] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:01.829111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:01.882839 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e999d-52fa-45c4-a14e-e957306b189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:01.883398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e999d-52fa-45c4-a14e-e957306b189e admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.883673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e999d-52fa-45c4-a14e-e957306b189e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.883939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e999d-52fa-45c4-a14e-e957306b189e admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.884625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e999d-52fa-45c4-a14e-e957306b189e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:01.886202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e999d-52fa-45c4-a14e-e957306b189e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:01.900083 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1272/2561] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:01.924753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91f8be-42a7-4d85-aa21-2dbdeba9963f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:01.924753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91f8be-42a7-4d85-aa21-2dbdeba9963f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:01.924753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:01.924753 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.925264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.925264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] PATH_INFO: `/v3/projects/9431661d673f4d3884c1b1bfb9a36856/users/69782c1ca83843cba684ac32e8ede079/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.925264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9431661d673f4d3884c1b1bfb9a36856, user_id=69782c1ca83843cba684ac32e8ede079, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:01.925264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:01.932534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1290/2562] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:01 2022] PUT /identity/v3/projects/9431661d673f4d3884c1b1bfb9a36856/users/69782c1ca83843cba684ac32e8ede079/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:01.943795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47a7cd-beb0-47c9-9bf9-fbd12a69aed6 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:01.944087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47a7cd-beb0-47c9-9bf9-fbd12a69aed6 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:01.944393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47a7cd-beb0-47c9-9bf9-fbd12a69aed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:01.990123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4e47a7cd-beb0-47c9-9bf9-fbd12a69aed6 None None] MFA Rules not processed for user `69782c1ca83843cba684ac32e8ede079`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:01.991392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4e47a7cd-beb0-47c9-9bf9-fbd12a69aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:02.069008 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1273/2563] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:02.080057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:02.085512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:02.162895 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cb556-0248-4bf0-bb3f-97ffa966d0aa admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:02.168048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d400a82-b2b2-42d1-88ae-ef08cf86baa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:02.168692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400a82-b2b2-42d1-88ae-ef08cf86baa1 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:02.168970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400a82-b2b2-42d1-88ae-ef08cf86baa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:02.169239 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400a82-b2b2-42d1-88ae-ef08cf86baa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:02.181788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d400a82-b2b2-42d1-88ae-ef08cf86baa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:02.183433 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d400a82-b2b2-42d1-88ae-ef08cf86baa1 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:02.194948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1291/2564] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:02 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:02.364943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47a7cd-beb0-47c9-9bf9-fbd12a69aed6 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:02.457231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47a7cd-beb0-47c9-9bf9-fbd12a69aed6 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:02.460874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:02.461516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:02.461797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:02.462069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] PATH_INFO: `/v3/users/8b5b9648afdf45859b2df3aba84a589d` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:02.463258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b5b9648afdf45859b2df3aba84a589d)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:02.465293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:02.733633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b5b9648afdf45859b2df3aba84a589d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:02.734377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b5b9648afdf45859b2df3aba84a589d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:02.746775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b5b9648afdf45859b2df3aba84a589d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:02.747553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b5b9648afdf45859b2df3aba84a589d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:02.822662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b5b9648afdf45859b2df3aba84a589d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:02.845895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1274/2565] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:02 2022] DELETE /identity/v3/users/8b5b9648afdf45859b2df3aba84a589d => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:02.850526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e310f5-1267-4d2a-ad0f-f405520dd4a0 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:02.851033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e310f5-1267-4d2a-ad0f-f405520dd4a0 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:02.851294 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e310f5-1267-4d2a-ad0f-f405520dd4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:02.872209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a6e310f5-1267-4d2a-ad0f-f405520dd4a0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:02.873428 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a6e310f5-1267-4d2a-ad0f-f405520dd4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:02.932340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1292/2566] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:02.943597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:02.945525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:03.007904 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e47b0f-a8f0-401a-ba4d-84137e80e46d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:03.011554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4766d-6004-4a44-be37-faaba642917b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:03.012188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4766d-6004-4a44-be37-faaba642917b service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:03.012460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4766d-6004-4a44-be37-faaba642917b service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:03.012725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4766d-6004-4a44-be37-faaba642917b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:03.022628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4766d-6004-4a44-be37-faaba642917b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:03.024122 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4766d-6004-4a44-be37-faaba642917b service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:03.034401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1275/2567] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:03.537369 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e310f5-1267-4d2a-ad0f-f405520dd4a0 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:03.539982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a6e310f5-1267-4d2a-ad0f-f405520dd4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:03.555413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e310f5-1267-4d2a-ad0f-f405520dd4a0 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:03.568322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:03.579328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:03.579754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:03.591968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] PATH_INFO: `/v3/projects/bad02b350b0148d2a017740cf79b35c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:03.594635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad02b350b0148d2a017740cf79b35c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:03.596956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:03.746588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad02b350b0148d2a017740cf79b35c5', 'request_id': 'req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:03.757863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad02b350b0148d2a017740cf79b35c5', 'request_id': 'req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:03.850600 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad02b350b0148d2a017740cf79b35c5', 'request_id': 'req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:03.863830 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] The token cache is being invalidate because project bad02b350b0148d2a017740cf79b35c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:03.864468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:03.865227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:03.867197 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1293/2568] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:03 2022] DELETE /identity/v3/projects/bad02b350b0148d2a017740cf79b35c5 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:03.872069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4766d-6004-4a44-be37-faaba642917b service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:03.886303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4766d-6004-4a44-be37-faaba642917b service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:03.951496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:03.952630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:03.952917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:03.953188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] PATH_INFO: `/v3/users/69782c1ca83843cba684ac32e8ede079` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:03.954442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69782c1ca83843cba684ac32e8ede079)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:03.956423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:04.164830 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:04.292637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69782c1ca83843cba684ac32e8ede079'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:04.293292 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69782c1ca83843cba684ac32e8ede079'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:04.305431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69782c1ca83843cba684ac32e8ede079'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:04.306074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69782c1ca83843cba684ac32e8ede079'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:04.410600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9373cef5-beb3-4a65-a5c3-22196eb4de21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69782c1ca83843cba684ac32e8ede079'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:04.424665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1276/2569] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:03 2022] DELETE /identity/v3/users/69782c1ca83843cba684ac32e8ede079 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:04.569523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:04.937589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:04.939067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:04.968977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8e286-c851-4965-bb2d-69d35e3c0ed8 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:05.042448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:05.043212 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:05.043533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:05.044518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] PATH_INFO: `/v3/projects/9431661d673f4d3884c1b1bfb9a36856` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:05.046357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9431661d673f4d3884c1b1bfb9a36856)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:05.048363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:05.337826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:05.352535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9431661d673f4d3884c1b1bfb9a36856', 'request_id': 'req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:05.359499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9431661d673f4d3884c1b1bfb9a36856', 'request_id': 'req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:05.431761 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9431661d673f4d3884c1b1bfb9a36856', 'request_id': 'req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:05.446903 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] The token cache is being invalidate because project 9431661d673f4d3884c1b1bfb9a36856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:05.447479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:05.448140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4ddd9b6c-7ffe-4944-be5c-91112b7b890b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:05.449692 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1294/2570] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:04 2022] DELETE /identity/v3/projects/9431661d673f4d3884c1b1bfb9a36856 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:05.455118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534a85-84b2-4d04-8165-7809439ceef7 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:05.456769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534a85-84b2-4d04-8165-7809439ceef7 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:05.457016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534a85-84b2-4d04-8165-7809439ceef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:05.483895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-71534a85-84b2-4d04-8165-7809439ceef7 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:05.485237 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-71534a85-84b2-4d04-8165-7809439ceef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:05.573423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1277/2571] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:05.581205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc484e2-a13b-439d-91aa-31f00f857c7b None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:05.581481 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc484e2-a13b-439d-91aa-31f00f857c7b None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:05.582188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc484e2-a13b-439d-91aa-31f00f857c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:05.605689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ffc484e2-a13b-439d-91aa-31f00f857c7b None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:05.608069 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ffc484e2-a13b-439d-91aa-31f00f857c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:05.618100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1295/2572] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:05.624085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71534a85-84b2-4d04-8165-7809439ceef7 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:05.635413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71534a85-84b2-4d04-8165-7809439ceef7 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:05.640859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:05.641788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:05.642125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:05.642455 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:05.643375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:05.643792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:05.645459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:05.659463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1278/2573] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:05:05.664458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc484e2-a13b-439d-91aa-31f00f857c7b None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:05.675891 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc484e2-a13b-439d-91aa-31f00f857c7b None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:05.682836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c4ba0-0685-4e61-bc09-a5bb39457be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:05.683695 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4ba0-0685-4e61-bc09-a5bb39457be8 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:05.684131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4ba0-0685-4e61-bc09-a5bb39457be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:05.684585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c4ba0-0685-4e61-bc09-a5bb39457be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:05.685935 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c4ba0-0685-4e61-bc09-a5bb39457be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:05.688085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c4ba0-0685-4e61-bc09-a5bb39457be8 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:05.835821 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1296/2574] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:05 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:05.839987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:05.854576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446c5bf-6f98-4279-972d-9b03af3d9975 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:05.928221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6fc967-0d84-45fa-a54c-a2d3af035849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:05.928969 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fc967-0d84-45fa-a54c-a2d3af035849 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:05.929253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fc967-0d84-45fa-a54c-a2d3af035849 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:05.929856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fc967-0d84-45fa-a54c-a2d3af035849 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:05.930667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6fc967-0d84-45fa-a54c-a2d3af035849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:05.933760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6fc967-0d84-45fa-a54c-a2d3af035849 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:06.144124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1279/2575] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:05 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:06.148323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c4ba0-0685-4e61-bc09-a5bb39457be8 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:06.159473 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c4ba0-0685-4e61-bc09-a5bb39457be8 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:06.163937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b834-6936-4e50-b1e7-70b748ab6a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:06.164584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b834-6936-4e50-b1e7-70b748ab6a71 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:06.164836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b834-6936-4e50-b1e7-70b748ab6a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:06.165098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b834-6936-4e50-b1e7-70b748ab6a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:06.165725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4b834-6936-4e50-b1e7-70b748ab6a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:06.167362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4b834-6936-4e50-b1e7-70b748ab6a71 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:06.178096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1297/2576] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:06.186318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6fc967-0d84-45fa-a54c-a2d3af035849 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:06.196648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6fc967-0d84-45fa-a54c-a2d3af035849 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:06.201780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276f564-de79-4767-9fcf-b446d882a7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:06.202392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276f564-de79-4767-9fcf-b446d882a7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:06.203498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276f564-de79-4767-9fcf-b446d882a7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:06.203498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276f564-de79-4767-9fcf-b446d882a7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:06.211226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276f564-de79-4767-9fcf-b446d882a7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:06.212855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276f564-de79-4767-9fcf-b446d882a7d6 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:06.224772 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1280/2577] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:06.229039 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b834-6936-4e50-b1e7-70b748ab6a71 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:06.240603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b834-6936-4e50-b1e7-70b748ab6a71 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:06.245181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876b725-96e7-4182-9848-52d271b329b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:06.245835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876b725-96e7-4182-9848-52d271b329b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:06.246107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876b725-96e7-4182-9848-52d271b329b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:06.246370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876b725-96e7-4182-9848-52d271b329b4 admin admin] PATH_INFO: `/v3/projects/87e5928f467a4fa2ad65fb8193eac84a/users/0818a4cbeda5480b8474de9c8b3875a9/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:06.261145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876b725-96e7-4182-9848-52d271b329b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e5928f467a4fa2ad65fb8193eac84a, user_id=0818a4cbeda5480b8474de9c8b3875a9, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:06.264169 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876b725-96e7-4182-9848-52d271b329b4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:06.343161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1298/2578] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:06 2022] PUT /identity/v3/projects/87e5928f467a4fa2ad65fb8193eac84a/users/0818a4cbeda5480b8474de9c8b3875a9/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:06.354842 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1483f2-ec45-41e3-a2f1-a5d9cb5e0865 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:06.355338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1483f2-ec45-41e3-a2f1-a5d9cb5e0865 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:06.355612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1483f2-ec45-41e3-a2f1-a5d9cb5e0865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:06.384596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4d1483f2-ec45-41e3-a2f1-a5d9cb5e0865 None None] MFA Rules not processed for user `0818a4cbeda5480b8474de9c8b3875a9`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:06.386124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4d1483f2-ec45-41e3-a2f1-a5d9cb5e0865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:06.460032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1281/2579] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:06.464469 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876b725-96e7-4182-9848-52d271b329b4 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:06.530543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876b725-96e7-4182-9848-52d271b329b4 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:06.536801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8ef3d-51f0-4513-ab52-4316bb1c2532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:06.536801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8ef3d-51f0-4513-ab52-4316bb1c2532 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:06.536801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8ef3d-51f0-4513-ab52-4316bb1c2532 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:06.536801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8ef3d-51f0-4513-ab52-4316bb1c2532 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:06.537205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8ef3d-51f0-4513-ab52-4316bb1c2532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:06.539529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8ef3d-51f0-4513-ab52-4316bb1c2532 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:06.720889 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1299/2580] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:06 2022] POST /identity/v3/projects => generated 363 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:06.724991 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1483f2-ec45-41e3-a2f1-a5d9cb5e0865 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:06.737207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1483f2-ec45-41e3-a2f1-a5d9cb5e0865 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:06.793918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf1057-22ad-454c-b54a-84214d795c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:06.795296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf1057-22ad-454c-b54a-84214d795c8f admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:06.795755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf1057-22ad-454c-b54a-84214d795c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:06.795944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf1057-22ad-454c-b54a-84214d795c8f admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:06.796635 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf1057-22ad-454c-b54a-84214d795c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:06.798527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf1057-22ad-454c-b54a-84214d795c8f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:06.942095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1282/2581] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:06 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:06.945440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8ef3d-51f0-4513-ab52-4316bb1c2532 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:06.955921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8ef3d-51f0-4513-ab52-4316bb1c2532 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:06.960589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:06.961166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:06.961442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:06.961714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:06.962353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:06.964045 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:06.975279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1300/2582] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:06.979620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf1057-22ad-454c-b54a-84214d795c8f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:06.991134 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf1057-22ad-454c-b54a-84214d795c8f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:06.998576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dcc1c2-28bd-4cf8-b058-5c3bd9337da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:06.999280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcc1c2-28bd-4cf8-b058-5c3bd9337da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:06.999604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcc1c2-28bd-4cf8-b058-5c3bd9337da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:06.999933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcc1c2-28bd-4cf8-b058-5c3bd9337da9 admin admin] PATH_INFO: `/v3/projects/548b7feed5644a969826ec1c07aad877/users/8b8f3b94a4bf48ee91dcfd77ac5885d1/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:07.004159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:07.005248 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:07.016536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dcc1c2-28bd-4cf8-b058-5c3bd9337da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=548b7feed5644a969826ec1c07aad877, user_id=8b8f3b94a4bf48ee91dcfd77ac5885d1, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:07.019268 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dcc1c2-28bd-4cf8-b058-5c3bd9337da9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:07.079237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41289b9-7ddd-47cf-9026-d86f2d5b4825 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:07.084250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cebaea-3cdc-40b1-86b6-0c4a7525adc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:07.084958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cebaea-3cdc-40b1-86b6-0c4a7525adc0 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:07.085301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cebaea-3cdc-40b1-86b6-0c4a7525adc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:07.085639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cebaea-3cdc-40b1-86b6-0c4a7525adc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:07.087643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-51cebaea-3cdc-40b1-86b6-0c4a7525adc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:07.099482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1283/2583] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:06 2022] PUT /identity/v3/projects/548b7feed5644a969826ec1c07aad877/users/8b8f3b94a4bf48ee91dcfd77ac5885d1/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:07.103208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dcc1c2-28bd-4cf8-b058-5c3bd9337da9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:07.176262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cebaea-3cdc-40b1-86b6-0c4a7525adc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:07.176925 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dcc1c2-28bd-4cf8-b058-5c3bd9337da9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:07.181913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cebaea-3cdc-40b1-86b6-0c4a7525adc0 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:07.182390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0850f2-739d-4ced-9bdd-b7a2cc36960c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:07.183264 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0850f2-739d-4ced-9bdd-b7a2cc36960c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:07.183608 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0850f2-739d-4ced-9bdd-b7a2cc36960c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:07.184045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0850f2-739d-4ced-9bdd-b7a2cc36960c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:07.184708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0850f2-739d-4ced-9bdd-b7a2cc36960c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:07.186838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0850f2-739d-4ced-9bdd-b7a2cc36960c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:07.198624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1284/2584] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:07.202149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1301/2585] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:07.203451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0850f2-739d-4ced-9bdd-b7a2cc36960c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:07.227087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0850f2-739d-4ced-9bdd-b7a2cc36960c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:07.235278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966afeb-44d5-4c8b-95f4-2cd19def1528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:07.236020 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966afeb-44d5-4c8b-95f4-2cd19def1528 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:07.236873 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966afeb-44d5-4c8b-95f4-2cd19def1528 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:07.237161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966afeb-44d5-4c8b-95f4-2cd19def1528 admin admin] PATH_INFO: `/v3/projects/548b7feed5644a969826ec1c07aad877/users/8b8f3b94a4bf48ee91dcfd77ac5885d1/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:07.243244 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966afeb-44d5-4c8b-95f4-2cd19def1528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=548b7feed5644a969826ec1c07aad877, user_id=8b8f3b94a4bf48ee91dcfd77ac5885d1, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:07.246239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966afeb-44d5-4c8b-95f4-2cd19def1528 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:07.265546 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1285/2586] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:07 2022] PUT /identity/v3/projects/548b7feed5644a969826ec1c07aad877/users/8b8f3b94a4bf48ee91dcfd77ac5885d1/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:07.276983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d9cf-e16a-4c49-b823-473a19e4a3b8 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:07.277729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d9cf-e16a-4c49-b823-473a19e4a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:07.278170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d9cf-e16a-4c49-b823-473a19e4a3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:07.321853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a3e6d9cf-e16a-4c49-b823-473a19e4a3b8 None None] MFA Rules not processed for user `8b8f3b94a4bf48ee91dcfd77ac5885d1`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:07.323339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a3e6d9cf-e16a-4c49-b823-473a19e4a3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:07.410503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:07.496724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:07.549804 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1302/2587] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:07.596136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966afeb-44d5-4c8b-95f4-2cd19def1528 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:07.688903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966afeb-44d5-4c8b-95f4-2cd19def1528 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:07.693283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:07.698671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:07.698671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:07.702776 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:07.714560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:07.716966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:07.734796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1286/2588] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:08.027106 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6d9cf-e16a-4c49-b823-473a19e4a3b8 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:08.039794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6d9cf-e16a-4c49-b823-473a19e4a3b8 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:08.043793 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:08.044806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:08.045142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:08.045468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:08.132660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:08.134351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:08.158284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1303/2589] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:09.171182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:09.473128 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:09.714215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:10.342186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:10.724809 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:11.404981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:14.550767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:14.552375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:14.619654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75774040-626d-4897-962b-d15099fe61cb service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:14.624339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:14.625044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:14.625313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:14.625583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:14.627391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:14.696374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:14.697825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:14.708599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1287/2590] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:18.822234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:18.823936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:18.839174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc95c2f-c0a5-45d3-8f4a-1e0b9b0d0d48 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:18.844684 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:18.845309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:18.845581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:18.845847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] PATH_INFO: `/v3/users/228837054cd648acb0df93dd423e1d83` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:18.847044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] RBAC: Authorizing `identity:delete_user(user_id=228837054cd648acb0df93dd423e1d83)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:18.848928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:19.054383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228837054cd648acb0df93dd423e1d83'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.055621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228837054cd648acb0df93dd423e1d83'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.069038 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228837054cd648acb0df93dd423e1d83'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.210982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228837054cd648acb0df93dd423e1d83'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.219060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-ea9d94a4-9c3f-4fe7-ba88-a6dff61becac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228837054cd648acb0df93dd423e1d83'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.225262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1304/2591] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:18 2022] DELETE /identity/v3/users/228837054cd648acb0df93dd423e1d83 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:19.504065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:19.514595 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96141c-b441-483e-9cbe-c040737d535f service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:19.602671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:19.603352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:19.603637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:19.603912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] PATH_INFO: `/v3/projects/e8bfb4199b0d4a57933fe1b685aa01cd` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:19.605072 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8bfb4199b0d4a57933fe1b685aa01cd)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:19.608241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:19.617264 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:19.743449 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8bfb4199b0d4a57933fe1b685aa01cd', 'request_id': 'req-67266cc1-ba00-48bc-836e-374c39154ba1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.750232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8bfb4199b0d4a57933fe1b685aa01cd', 'request_id': 'req-67266cc1-ba00-48bc-836e-374c39154ba1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.827413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8bfb4199b0d4a57933fe1b685aa01cd', 'request_id': 'req-67266cc1-ba00-48bc-836e-374c39154ba1'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.850024 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] The token cache is being invalidate because project e8bfb4199b0d4a57933fe1b685aa01cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:19.850309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.851027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-67266cc1-ba00-48bc-836e-374c39154ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:19.852536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1288/2592] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:19 2022] DELETE /identity/v3/projects/e8bfb4199b0d4a57933fe1b685aa01cd => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:19.857549 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702dcca-df11-467a-8344-997ecb777088 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:19.858036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702dcca-df11-467a-8344-997ecb777088 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:19.858147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702dcca-df11-467a-8344-997ecb777088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:19.877961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7702dcca-df11-467a-8344-997ecb777088 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:19.879185 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7702dcca-df11-467a-8344-997ecb777088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:19.903574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:19.938181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1305/2593] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:19.946440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46717d-ba95-426f-8b1a-315c81f9209d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:19.946716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46717d-ba95-426f-8b1a-315c81f9209d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:19.947013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46717d-ba95-426f-8b1a-315c81f9209d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:19.967410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5d46717d-ba95-426f-8b1a-315c81f9209d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:19.968744 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5d46717d-ba95-426f-8b1a-315c81f9209d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:19.977829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1289/2594] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:19.982153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702dcca-df11-467a-8344-997ecb777088 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:19.992008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702dcca-df11-467a-8344-997ecb777088 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:19.996618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:19.998020 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:19.998285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:19.998404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:19.999028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:19.999328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:20.000915 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:20.013596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1306/2595] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:05:20.017981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46717d-ba95-426f-8b1a-315c81f9209d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:20.027351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46717d-ba95-426f-8b1a-315c81f9209d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:20.032299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbde5d-c776-4fbc-ac85-c331d4b6b3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:20.032885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbde5d-c776-4fbc-ac85-c331d4b6b3ab admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:20.033156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbde5d-c776-4fbc-ac85-c331d4b6b3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:20.033422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbde5d-c776-4fbc-ac85-c331d4b6b3ab admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:20.034034 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fbde5d-c776-4fbc-ac85-c331d4b6b3ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:20.035981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fbde5d-c776-4fbc-ac85-c331d4b6b3ab admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:20.137132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1290/2596] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:20 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:20.142542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:20.151725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d911d50-98fd-4768-b619-17b8a3fa4805 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:20.213648 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0342-8319-48e3-a345-af771d35532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:20.214264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0342-8319-48e3-a345-af771d35532c admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:20.214540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0342-8319-48e3-a345-af771d35532c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:20.214807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0342-8319-48e3-a345-af771d35532c admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:20.215472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e0342-8319-48e3-a345-af771d35532c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:20.217854 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e0342-8319-48e3-a345-af771d35532c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:20.336319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1307/2597] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:20 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:20.340326 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbde5d-c776-4fbc-ac85-c331d4b6b3ab admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:20.350893 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbde5d-c776-4fbc-ac85-c331d4b6b3ab admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:20.355802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ae60c-20a5-44d2-a190-12dafa157f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:20.357353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae60c-20a5-44d2-a190-12dafa157f16 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:20.357735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae60c-20a5-44d2-a190-12dafa157f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:20.358010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ae60c-20a5-44d2-a190-12dafa157f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:20.358702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ae60c-20a5-44d2-a190-12dafa157f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:20.360779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ae60c-20a5-44d2-a190-12dafa157f16 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:20.373038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1291/2598] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:20.377128 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0342-8319-48e3-a345-af771d35532c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:20.387279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0342-8319-48e3-a345-af771d35532c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:20.392078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5b4e8-3111-4795-b11c-1f234810e321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:20.392671 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b4e8-3111-4795-b11c-1f234810e321 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:20.392953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b4e8-3111-4795-b11c-1f234810e321 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:20.393224 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5b4e8-3111-4795-b11c-1f234810e321 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:20.393861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5b4e8-3111-4795-b11c-1f234810e321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:20.395488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5b4e8-3111-4795-b11c-1f234810e321 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:20.406170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1308/2599] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:20.410240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ae60c-20a5-44d2-a190-12dafa157f16 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:20.424524 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ae60c-20a5-44d2-a190-12dafa157f16 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:20.429255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:20.429912 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:20.430190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:20.430466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] PATH_INFO: `/v3/projects/cbf13aeb88f04e42af5e50a95d30d717/users/192d31bb52124ac5bcd11feda094590f/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:20.446303 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf13aeb88f04e42af5e50a95d30d717, user_id=192d31bb52124ac5bcd11feda094590f, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:20.449027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:20.536385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1292/2600] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:20 2022] PUT /identity/v3/projects/cbf13aeb88f04e42af5e50a95d30d717/users/192d31bb52124ac5bcd11feda094590f/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:20.548580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cca04-f9e0-41a8-970d-d95ebc4aeecd None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:20.553098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cca04-f9e0-41a8-970d-d95ebc4aeecd None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:20.553098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cca04-f9e0-41a8-970d-d95ebc4aeecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:20.594615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6c8cca04-f9e0-41a8-970d-d95ebc4aeecd None None] MFA Rules not processed for user `192d31bb52124ac5bcd11feda094590f`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:20.595868 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6c8cca04-f9e0-41a8-970d-d95ebc4aeecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:20.670964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1309/2601] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:20.680721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:20.682631 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:20.763994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322a3b1-47fc-4143-af08-8347ee2e3851 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:20.769632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2607597-5fe0-4638-9c27-13698394fe61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:20.770242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2607597-5fe0-4638-9c27-13698394fe61 service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:20.770520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2607597-5fe0-4638-9c27-13698394fe61 service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:20.770792 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2607597-5fe0-4638-9c27-13698394fe61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:20.779032 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2607597-5fe0-4638-9c27-13698394fe61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:20.780806 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2607597-5fe0-4638-9c27-13698394fe61 service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:20.789732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1293/2602] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:21.010393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cca04-f9e0-41a8-970d-d95ebc4aeecd None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:21.038750 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:21.073972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cca04-f9e0-41a8-970d-d95ebc4aeecd None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:21.079658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:21.080313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:21.080586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:21.080853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] PATH_INFO: `/v3/users/192d31bb52124ac5bcd11feda094590f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:21.081961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=192d31bb52124ac5bcd11feda094590f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:21.084339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:21.328693 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192d31bb52124ac5bcd11feda094590f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:21.329423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192d31bb52124ac5bcd11feda094590f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:21.342203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192d31bb52124ac5bcd11feda094590f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:21.455629 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192d31bb52124ac5bcd11feda094590f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:21.463628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192d31bb52124ac5bcd11feda094590f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:21.470281 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1310/2603] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:21 2022] DELETE /identity/v3/users/192d31bb52124ac5bcd11feda094590f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:21.474829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dac8b-1733-484b-8901-0b1f18c3fa5f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:21.475203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dac8b-1733-484b-8901-0b1f18c3fa5f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:21.475577 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dac8b-1733-484b-8901-0b1f18c3fa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:21.494677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c86dac8b-1733-484b-8901-0b1f18c3fa5f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:21.495993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c86dac8b-1733-484b-8901-0b1f18c3fa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:21.509036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:21.556285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1294/2604] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:21.566642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:21.568208 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:21.628008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee95726-8a04-464a-a134-6e22eafce39d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:21.631605 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ff972-1b1d-40d2-b685-d158999449e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:21.632204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ff972-1b1d-40d2-b685-d158999449e8 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:21.632475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ff972-1b1d-40d2-b685-d158999449e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:21.632737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ff972-1b1d-40d2-b685-d158999449e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:21.641799 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ff972-1b1d-40d2-b685-d158999449e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:21.643255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ff972-1b1d-40d2-b685-d158999449e8 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:21.653054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1311/2605] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:22.144248 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dac8b-1733-484b-8901-0b1f18c3fa5f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:22.145759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c86dac8b-1733-484b-8901-0b1f18c3fa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:22.159491 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86dac8b-1733-484b-8901-0b1f18c3fa5f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:22.165276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:22.166087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:22.166367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:22.166633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] PATH_INFO: `/v3/projects/cbf13aeb88f04e42af5e50a95d30d717` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:22.168516 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf13aeb88f04e42af5e50a95d30d717)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:22.177682 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:22.370976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf13aeb88f04e42af5e50a95d30d717', 'request_id': 'req-7fbdf16a-122a-4243-bd95-c77432392e1d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:22.377505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf13aeb88f04e42af5e50a95d30d717', 'request_id': 'req-7fbdf16a-122a-4243-bd95-c77432392e1d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:22.489565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf13aeb88f04e42af5e50a95d30d717', 'request_id': 'req-7fbdf16a-122a-4243-bd95-c77432392e1d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:22.502569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] The token cache is being invalidate because project cbf13aeb88f04e42af5e50a95d30d717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:22.504214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:22.504214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7fbdf16a-122a-4243-bd95-c77432392e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:22.505052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1295/2606] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:22 2022] DELETE /identity/v3/projects/cbf13aeb88f04e42af5e50a95d30d717 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:22.510124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca5a2b-07b7-4242-9646-3473a8afdc00 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:22.510637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca5a2b-07b7-4242-9646-3473a8afdc00 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:22.512208 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca5a2b-07b7-4242-9646-3473a8afdc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:22.532219 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b7ca5a2b-07b7-4242-9646-3473a8afdc00 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:22.533426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b7ca5a2b-07b7-4242-9646-3473a8afdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:22.579089 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:22.596351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1312/2607] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:22.604262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab34fc-48ee-4d93-83c5-aa9a577e0bc3 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:22.604779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab34fc-48ee-4d93-83c5-aa9a577e0bc3 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:22.604779 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab34fc-48ee-4d93-83c5-aa9a577e0bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:22.626670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-45ab34fc-48ee-4d93-83c5-aa9a577e0bc3 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:22.627962 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-45ab34fc-48ee-4d93-83c5-aa9a577e0bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:22.635998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1296/2608] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:22.640767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca5a2b-07b7-4242-9646-3473a8afdc00 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:22.650582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca5a2b-07b7-4242-9646-3473a8afdc00 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:22.655538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:22.656561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:22.656838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:22.657104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:22.657777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:22.658067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:22.659681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:22.672274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1313/2609] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:05:22.676240 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab34fc-48ee-4d93-83c5-aa9a577e0bc3 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:22.685478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab34fc-48ee-4d93-83c5-aa9a577e0bc3 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:22.692931 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02b697-d28b-4371-8ad0-bc7601789a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:22.693527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02b697-d28b-4371-8ad0-bc7601789a66 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:22.693797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02b697-d28b-4371-8ad0-bc7601789a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:22.694064 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02b697-d28b-4371-8ad0-bc7601789a66 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:22.694675 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02b697-d28b-4371-8ad0-bc7601789a66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:22.696623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02b697-d28b-4371-8ad0-bc7601789a66 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:22.821685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1297/2610] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:22 2022] POST /identity/v3/projects => generated 409 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:22.826034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:22.836237 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7cda0-5da9-4d90-a806-f8d192ad799f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:22.893067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b3afa-c8c0-40fe-b2e9-a25874a3a867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:22.893665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3afa-c8c0-40fe-b2e9-a25874a3a867 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:22.893939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3afa-c8c0-40fe-b2e9-a25874a3a867 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:22.894203 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3afa-c8c0-40fe-b2e9-a25874a3a867 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:22.894808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b3afa-c8c0-40fe-b2e9-a25874a3a867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:22.896725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b3afa-c8c0-40fe-b2e9-a25874a3a867 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:22.998322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1314/2611] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:22 2022] POST /identity/v3/users => generated 305 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:23.002745 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02b697-d28b-4371-8ad0-bc7601789a66 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:23.014825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02b697-d28b-4371-8ad0-bc7601789a66 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:23.018651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968ffd9-bcaa-4ece-a814-377c2eae0f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:23.019347 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968ffd9-bcaa-4ece-a814-377c2eae0f70 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:23.019661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968ffd9-bcaa-4ece-a814-377c2eae0f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:23.019950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968ffd9-bcaa-4ece-a814-377c2eae0f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:23.020742 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968ffd9-bcaa-4ece-a814-377c2eae0f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:23.022191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968ffd9-bcaa-4ece-a814-377c2eae0f70 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:23.036235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1298/2612] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:23.040223 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b3afa-c8c0-40fe-b2e9-a25874a3a867 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:23.051487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b3afa-c8c0-40fe-b2e9-a25874a3a867 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:23.059759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44865dc0-895a-4782-9b86-61cba2ebb4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:23.059759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44865dc0-895a-4782-9b86-61cba2ebb4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:23.059759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44865dc0-895a-4782-9b86-61cba2ebb4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:23.059759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44865dc0-895a-4782-9b86-61cba2ebb4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:23.060312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44865dc0-895a-4782-9b86-61cba2ebb4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:23.060312 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44865dc0-895a-4782-9b86-61cba2ebb4e0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:23.072295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1315/2613] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:23.076778 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968ffd9-bcaa-4ece-a814-377c2eae0f70 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:23.088541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968ffd9-bcaa-4ece-a814-377c2eae0f70 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:23.094144 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca015a9-35b3-48c4-8c5b-1e6a3fe527ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:23.094807 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca015a9-35b3-48c4-8c5b-1e6a3fe527ef admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:23.095109 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca015a9-35b3-48c4-8c5b-1e6a3fe527ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:23.095388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca015a9-35b3-48c4-8c5b-1e6a3fe527ef admin admin] PATH_INFO: `/v3/projects/92273ef6162a46a4a6c791d282b360aa/users/ba674a10ca524b47b49130a74d29c33d/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:23.111549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca015a9-35b3-48c4-8c5b-1e6a3fe527ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=92273ef6162a46a4a6c791d282b360aa, user_id=ba674a10ca524b47b49130a74d29c33d, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:23.113311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca015a9-35b3-48c4-8c5b-1e6a3fe527ef admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:23.199410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1299/2614] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:23 2022] PUT /identity/v3/projects/92273ef6162a46a4a6c791d282b360aa/users/ba674a10ca524b47b49130a74d29c33d/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:23.210824 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0efb9a-c3bb-412f-bf4a-3971ee8d6b74 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:23.211138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0efb9a-c3bb-412f-bf4a-3971ee8d6b74 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:23.211506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0efb9a-c3bb-412f-bf4a-3971ee8d6b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:23.243233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7c0efb9a-c3bb-412f-bf4a-3971ee8d6b74 None None] MFA Rules not processed for user `ba674a10ca524b47b49130a74d29c33d`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:23.245181 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7c0efb9a-c3bb-412f-bf4a-3971ee8d6b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:23.320421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1316/2615] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:23.324924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca015a9-35b3-48c4-8c5b-1e6a3fe527ef admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:23.366960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:23.388305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca015a9-35b3-48c4-8c5b-1e6a3fe527ef admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:23.393294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f89bb-3a94-4dd4-9bcb-41c5e3cffca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:23.393975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f89bb-3a94-4dd4-9bcb-41c5e3cffca1 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:23.394199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f89bb-3a94-4dd4-9bcb-41c5e3cffca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:23.394465 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f89bb-3a94-4dd4-9bcb-41c5e3cffca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:23.395114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f89bb-3a94-4dd4-9bcb-41c5e3cffca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:23.397010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f89bb-3a94-4dd4-9bcb-41c5e3cffca1 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:23.523255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1300/2616] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:23 2022] POST /identity/v3/projects => generated 407 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:23.524863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0efb9a-c3bb-412f-bf4a-3971ee8d6b74 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:23.537628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0efb9a-c3bb-412f-bf4a-3971ee8d6b74 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:23.592522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:23.612249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50b290-421c-46c5-8724-da777deafd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:23.612950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b290-421c-46c5-8724-da777deafd91 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:23.613227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b290-421c-46c5-8724-da777deafd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:23.613639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b290-421c-46c5-8724-da777deafd91 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:23.614276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50b290-421c-46c5-8724-da777deafd91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:23.616308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50b290-421c-46c5-8724-da777deafd91 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:23.752956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1317/2617] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:23 2022] POST /identity/v3/users => generated 310 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:23.757037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f89bb-3a94-4dd4-9bcb-41c5e3cffca1 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:23.768639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f89bb-3a94-4dd4-9bcb-41c5e3cffca1 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:23.773264 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd427-e7e1-4077-bdbc-36cc506c8b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:23.773867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd427-e7e1-4077-bdbc-36cc506c8b4c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:23.774280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd427-e7e1-4077-bdbc-36cc506c8b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:23.774463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd427-e7e1-4077-bdbc-36cc506c8b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:23.775397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efd427-e7e1-4077-bdbc-36cc506c8b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:23.777886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efd427-e7e1-4077-bdbc-36cc506c8b4c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:23.788922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1301/2618] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:23.793572 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50b290-421c-46c5-8724-da777deafd91 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:23.804930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50b290-421c-46c5-8724-da777deafd91 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:23.809634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2d57f-801a-4866-9b59-711c3a9a34a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:23.810280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2d57f-801a-4866-9b59-711c3a9a34a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:23.812221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2d57f-801a-4866-9b59-711c3a9a34a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:23.812221 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2d57f-801a-4866-9b59-711c3a9a34a2 admin admin] PATH_INFO: `/v3/projects/609083a2099b4670a5dbf14172eb4dd7/users/e5da3d4abcfc4dd8953e22d24acb835c/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:23.826662 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2d57f-801a-4866-9b59-711c3a9a34a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609083a2099b4670a5dbf14172eb4dd7, user_id=e5da3d4abcfc4dd8953e22d24acb835c, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:23.830548 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2d57f-801a-4866-9b59-711c3a9a34a2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:23.930460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1318/2619] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:23 2022] PUT /identity/v3/projects/609083a2099b4670a5dbf14172eb4dd7/users/e5da3d4abcfc4dd8953e22d24acb835c/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:23.934666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd427-e7e1-4077-bdbc-36cc506c8b4c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:23.948089 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd427-e7e1-4077-bdbc-36cc506c8b4c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:24.004589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbdeac-017c-4aab-9da9-0950ba9fa3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:24.005211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbdeac-017c-4aab-9da9-0950ba9fa3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:24.005490 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbdeac-017c-4aab-9da9-0950ba9fa3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:24.005753 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbdeac-017c-4aab-9da9-0950ba9fa3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:24.006380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbdeac-017c-4aab-9da9-0950ba9fa3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:24.007984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbdeac-017c-4aab-9da9-0950ba9fa3d2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:24.018306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1302/2620] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:24.022538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2d57f-801a-4866-9b59-711c3a9a34a2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:24.033298 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2d57f-801a-4866-9b59-711c3a9a34a2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:24.037905 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:24.038547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:24.038820 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:24.039119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] PATH_INFO: `/v3/projects/609083a2099b4670a5dbf14172eb4dd7/users/e5da3d4abcfc4dd8953e22d24acb835c/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:24.041046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609083a2099b4670a5dbf14172eb4dd7, user_id=e5da3d4abcfc4dd8953e22d24acb835c, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:24.043786 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:24.053916 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1319/2621] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:24 2022] PUT /identity/v3/projects/609083a2099b4670a5dbf14172eb4dd7/users/e5da3d4abcfc4dd8953e22d24acb835c/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:24.065927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd48f0-59be-4daf-909b-0a5914b37985 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:24.066234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd48f0-59be-4daf-909b-0a5914b37985 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:24.068783 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd48f0-59be-4daf-909b-0a5914b37985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:24.096489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-2efd48f0-59be-4daf-909b-0a5914b37985 None None] MFA Rules not processed for user `e5da3d4abcfc4dd8953e22d24acb835c`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:24.098191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2efd48f0-59be-4daf-909b-0a5914b37985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:24.166480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1303/2622] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:24 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:24.178532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:24.180309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:24.238475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ef277-49b5-43a4-9cf9-3cf6fd142ff3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:24.241960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffe42c-ec6b-4142-9779-17ce8c78abdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:24.242550 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffe42c-ec6b-4142-9779-17ce8c78abdb service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:24.242822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffe42c-ec6b-4142-9779-17ce8c78abdb service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:24.243115 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffe42c-ec6b-4142-9779-17ce8c78abdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:24.252104 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffe42c-ec6b-4142-9779-17ce8c78abdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:24.253508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffe42c-ec6b-4142-9779-17ce8c78abdb service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:24.263351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1320/2623] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:24 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:24.621545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd48f0-59be-4daf-909b-0a5914b37985 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:24.631411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd48f0-59be-4daf-909b-0a5914b37985 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:24.634980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:24.635579 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:24.635849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:24.636112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:24.671112 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:24.698413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:24.700021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:24.709435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1304/2624] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:24 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:30.024012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed317b5-7ec6-46bc-8bcf-f7e6bf790d8e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:30.024522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed317b5-7ec6-46bc-8bcf-f7e6bf790d8e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:30.024522 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed317b5-7ec6-46bc-8bcf-f7e6bf790d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:30.043276 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-eed317b5-7ec6-46bc-8bcf-f7e6bf790d8e None None] MFA Rules not processed for user `c911a9324cb744a6a1aeb068565d3f89`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:30.045730 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-eed317b5-7ec6-46bc-8bcf-f7e6bf790d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:30.100234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1321/2625] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:34.753427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=13 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:38.237390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:38.239058 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:38.305351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468366-0179-40f4-ac4a-a5cadbeceb67 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:38.310574 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:38.311267 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:38.311556 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:38.311825 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] PATH_INFO: `/v3/users/0818a4cbeda5480b8474de9c8b3875a9` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:38.312995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0818a4cbeda5480b8474de9c8b3875a9)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:38.315331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:38.710373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0818a4cbeda5480b8474de9c8b3875a9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:38.711164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0818a4cbeda5480b8474de9c8b3875a9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:38.724219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0818a4cbeda5480b8474de9c8b3875a9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:38.724966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0818a4cbeda5480b8474de9c8b3875a9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:38.855235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0818a4cbeda5480b8474de9c8b3875a9'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:38.873403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1305/2626] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:38 2022] DELETE /identity/v3/users/0818a4cbeda5480b8474de9c8b3875a9 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:38.877817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c3e09-c4f3-40fa-b23d-1ba675e0f74a None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:38.878340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c3e09-c4f3-40fa-b23d-1ba675e0f74a None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:38.878446 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c3e09-c4f3-40fa-b23d-1ba675e0f74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:38.903032 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7f6c3e09-c4f3-40fa-b23d-1ba675e0f74a None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:38.904727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7f6c3e09-c4f3-40fa-b23d-1ba675e0f74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:38.958023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:38.984196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1322/2627] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:38.995149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:38.996924 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:39.084159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a50a82-818b-499c-b10c-299a8391fad9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:39.089230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:39.090117 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:39.090468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:39.090830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:39.103729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:39.105896 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:39.119693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1306/2628] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:39.730526 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c3e09-c4f3-40fa-b23d-1ba675e0f74a None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:39.732034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7f6c3e09-c4f3-40fa-b23d-1ba675e0f74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:39.744744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c3e09-c4f3-40fa-b23d-1ba675e0f74a None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:39.755808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:39.757835 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:39.758116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:39.762201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] PATH_INFO: `/v3/projects/87e5928f467a4fa2ad65fb8193eac84a` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:39.762201 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e5928f467a4fa2ad65fb8193eac84a)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:39.767742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:39.768550 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:39.770188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:39.789456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567959b-88a8-4195-928d-0bc1e092c32b service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:39.793799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:39.794435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:39.794706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:39.795190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] PATH_INFO: `/v3/users/ba674a10ca524b47b49130a74d29c33d` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:39.796385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba674a10ca524b47b49130a74d29c33d)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:39.798317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:39.816837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:39.923956 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e5928f467a4fa2ad65fb8193eac84a', 'request_id': 'req-4b3d178d-c62a-402a-89ad-194309b9df71'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:39.929805 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e5928f467a4fa2ad65fb8193eac84a', 'request_id': 'req-4b3d178d-c62a-402a-89ad-194309b9df71'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.021172 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e5928f467a4fa2ad65fb8193eac84a', 'request_id': 'req-4b3d178d-c62a-402a-89ad-194309b9df71'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.033860 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] The token cache is being invalidate because project 87e5928f467a4fa2ad65fb8193eac84a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:40.034206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.034831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.036336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1323/2629] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:39 2022] DELETE /identity/v3/projects/87e5928f467a4fa2ad65fb8193eac84a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:40.049292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:40.049292 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:40.107664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d178d-c62a-402a-89ad-194309b9df71 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:40.113044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:40.113773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:40.114126 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:40.114464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] PATH_INFO: `/v3/users/8b8f3b94a4bf48ee91dcfd77ac5885d1` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:40.116340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b8f3b94a4bf48ee91dcfd77ac5885d1)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:40.116976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba674a10ca524b47b49130a74d29c33d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.117639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba674a10ca524b47b49130a74d29c33d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.118599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:40.144175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba674a10ca524b47b49130a74d29c33d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.144939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba674a10ca524b47b49130a74d29c33d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.215469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-24b2a6b2-bdc2-401b-a381-c8d1c1030435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba674a10ca524b47b49130a74d29c33d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.228995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1307/2630] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:39 2022] DELETE /identity/v3/users/ba674a10ca524b47b49130a74d29c33d => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:40.233661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e626fb-0eff-4522-9b8a-cd34cc9555b0 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:40.233943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e626fb-0eff-4522-9b8a-cd34cc9555b0 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:40.234297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e626fb-0eff-4522-9b8a-cd34cc9555b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:40.254442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-d8e626fb-0eff-4522-9b8a-cd34cc9555b0 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:40.255680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d8e626fb-0eff-4522-9b8a-cd34cc9555b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:40.326164 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1308/2631] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:40.333853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b8f3b94a4bf48ee91dcfd77ac5885d1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.344691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b8f3b94a4bf48ee91dcfd77ac5885d1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.345027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e626fb-0eff-4522-9b8a-cd34cc9555b0 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:40.345027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d8e626fb-0eff-4522-9b8a-cd34cc9555b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:40.351455 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b8f3b94a4bf48ee91dcfd77ac5885d1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.356965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:40.398764 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e626fb-0eff-4522-9b8a-cd34cc9555b0 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:40.402697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:40.403338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:40.403609 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:40.403874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:40.413295 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:40.414723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:40.424688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1309/2632] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:40.494738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b8f3b94a4bf48ee91dcfd77ac5885d1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.502147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b8f3b94a4bf48ee91dcfd77ac5885d1'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:40.508839 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1324/2633] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:40 2022] DELETE /identity/v3/users/8b8f3b94a4bf48ee91dcfd77ac5885d1 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:40.874673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:40.876201 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:40.905510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b265fd-c566-451b-b46c-a396c35649e5 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:40.957796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:40.982318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:40.982981 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:40.983256 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:40.983523 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] PATH_INFO: `/v3/projects/92273ef6162a46a4a6c791d282b360aa` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:40.984702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] RBAC: Authorizing `identity:delete_project(project_id=92273ef6162a46a4a6c791d282b360aa)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:40.986681 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:41.039161 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:41.040776 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:41.057712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a35e9e-4584-40e8-820f-a66a7d5ae644 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:41.063004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:41.063755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:41.068052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:41.068052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] PATH_INFO: `/v3/projects/548b7feed5644a969826ec1c07aad877` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:41.068052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=548b7feed5644a969826ec1c07aad877)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:41.068052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:41.158766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92273ef6162a46a4a6c791d282b360aa', 'request_id': 'req-5af34fde-04cd-4dc2-ae0e-67905a72fbde'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.164601 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92273ef6162a46a4a6c791d282b360aa', 'request_id': 'req-5af34fde-04cd-4dc2-ae0e-67905a72fbde'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.273087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '548b7feed5644a969826ec1c07aad877', 'request_id': 'req-dc2e7760-fbb9-4743-b1f7-30b3add23159'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.278802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '548b7feed5644a969826ec1c07aad877', 'request_id': 'req-dc2e7760-fbb9-4743-b1f7-30b3add23159'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.285389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92273ef6162a46a4a6c791d282b360aa', 'request_id': 'req-5af34fde-04cd-4dc2-ae0e-67905a72fbde'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.296890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] The token cache is being invalidate because project 92273ef6162a46a4a6c791d282b360aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:41.297173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.297878 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.299549 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1310/2634] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:40 2022] DELETE /identity/v3/projects/92273ef6162a46a4a6c791d282b360aa => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:41.308714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:41.310124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:41.374650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af34fde-04cd-4dc2-ae0e-67905a72fbde admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:41.408955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '548b7feed5644a969826ec1c07aad877', 'request_id': 'req-dc2e7760-fbb9-4743-b1f7-30b3add23159'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.421537 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] The token cache is being invalidate because project 548b7feed5644a969826ec1c07aad877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:41.421825 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.422471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-dc2e7760-fbb9-4743-b1f7-30b3add23159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.423969 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1325/2635] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:41 2022] DELETE /identity/v3/projects/548b7feed5644a969826ec1c07aad877 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:41.429703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64d7a-a689-4401-b434-d670179df47e None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:41.430063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64d7a-a689-4401-b434-d670179df47e None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:41.430332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64d7a-a689-4401-b434-d670179df47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:41.431425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:41.432075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:41.432355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:41.432627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] PATH_INFO: `/v3/users/e5da3d4abcfc4dd8953e22d24acb835c` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:41.433758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5da3d4abcfc4dd8953e22d24acb835c)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:41.435767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:41.451406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6cc64d7a-a689-4401-b434-d670179df47e None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:41.452628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6cc64d7a-a689-4401-b434-d670179df47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:41.464613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1326/2636] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:41.471183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97031ad-a62e-4736-bba3-d8f4dc8d4890 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:41.471551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97031ad-a62e-4736-bba3-d8f4dc8d4890 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:41.471881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97031ad-a62e-4736-bba3-d8f4dc8d4890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:41.492561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-a97031ad-a62e-4736-bba3-d8f4dc8d4890 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:41.493778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-a97031ad-a62e-4736-bba3-d8f4dc8d4890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:41.501708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1327/2637] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:41.505934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97031ad-a62e-4736-bba3-d8f4dc8d4890 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:41.517745 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97031ad-a62e-4736-bba3-d8f4dc8d4890 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:41.525432 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:41.526052 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:41.526327 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:41.526589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:41.527274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:41.527567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:41.529178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:41.544669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1328/2638] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 16:05:41.547396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:41.558286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89ea24-9814-4ce8-8b12-bac811271f01 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:41.563347 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50119c-f646-4956-bbd1-71aba6472ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:41.563972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50119c-f646-4956-bbd1-71aba6472ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:41.564247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50119c-f646-4956-bbd1-71aba6472ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:41.564515 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50119c-f646-4956-bbd1-71aba6472ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:41.565426 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50119c-f646-4956-bbd1-71aba6472ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:41.567417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50119c-f646-4956-bbd1-71aba6472ce2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:41.740050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1329/2639] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:41 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:41.741982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50119c-f646-4956-bbd1-71aba6472ce2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:41.817472 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50119c-f646-4956-bbd1-71aba6472ce2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:41.822845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:41.823875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:41.823875 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:41.824065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:41.824673 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:41.826634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:41.826769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5da3d4abcfc4dd8953e22d24acb835c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.827392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5da3d4abcfc4dd8953e22d24acb835c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.841069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5da3d4abcfc4dd8953e22d24acb835c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.841837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5da3d4abcfc4dd8953e22d24acb835c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.923655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5da3d4abcfc4dd8953e22d24acb835c'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:41.931021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1330/2640] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:41 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:41.937476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:41.937700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1311/2641] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:41 2022] DELETE /identity/v3/users/e5da3d4abcfc4dd8953e22d24acb835c => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:41.939491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:41.947198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:41.954364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6724b93-cdee-4adf-86b6-a1ec5ffe80dc admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:41.959199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:42.011173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d684c3-f8b1-4c71-bded-8fc869736f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:42.011924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d684c3-f8b1-4c71-bded-8fc869736f63 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.012195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d684c3-f8b1-4c71-bded-8fc869736f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.012457 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d684c3-f8b1-4c71-bded-8fc869736f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.013085 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d684c3-f8b1-4c71-bded-8fc869736f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:42.014664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d684c3-f8b1-4c71-bded-8fc869736f63 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:42.024976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1331/2642] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:42.029397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:42.041781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349c3a6-f5b8-4359-b88a-4d10b442015e admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:42.047006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:42.047676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.047940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.048214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.048868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:42.050476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:42.064586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1312/2643] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:42.077912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d684c3-f8b1-4c71-bded-8fc869736f63 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:42.077912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d684c3-f8b1-4c71-bded-8fc869736f63 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:42.082415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e2ce8-ff4b-4ec8-9be2-370576ee61db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:42.083107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e2ce8-ff4b-4ec8-9be2-370576ee61db admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.083387 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e2ce8-ff4b-4ec8-9be2-370576ee61db admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.083659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e2ce8-ff4b-4ec8-9be2-370576ee61db admin admin] PATH_INFO: `/v3/projects/ef490e4ac5d0483ab0f446e5d46355d1/users/a78e9f89d5a5490ba236545b8d95a851/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.098877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e2ce8-ff4b-4ec8-9be2-370576ee61db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef490e4ac5d0483ab0f446e5d46355d1, user_id=a78e9f89d5a5490ba236545b8d95a851, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:42.102050 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e2ce8-ff4b-4ec8-9be2-370576ee61db admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:42.225923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:42.228162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:42.241980 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191eb81-fa90-45cf-a359-eef3ea26b7be admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:42.246710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:42.247494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.247708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.247984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] PATH_INFO: `/v3/users/f074952b3564456abe94dcd0be50531d` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.261167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f074952b3564456abe94dcd0be50531d)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:42.261167 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:42.290712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1332/2644] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:42 2022] PUT /identity/v3/projects/ef490e4ac5d0483ab0f446e5d46355d1/users/a78e9f89d5a5490ba236545b8d95a851/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:42.301004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c8f25-27ad-4171-b223-a0664d9c2673 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.301289 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c8f25-27ad-4171-b223-a0664d9c2673 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.301560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c8f25-27ad-4171-b223-a0664d9c2673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.332131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-b33c8f25-27ad-4171-b223-a0664d9c2673 None None] MFA Rules not processed for user `a78e9f89d5a5490ba236545b8d95a851`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:42.332858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b33c8f25-27ad-4171-b223-a0664d9c2673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:42.403395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1333/2645] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:42.407885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c8f25-27ad-4171-b223-a0664d9c2673 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:42.469835 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f074952b3564456abe94dcd0be50531d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:42.470567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f074952b3564456abe94dcd0be50531d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:42.474303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c8f25-27ad-4171-b223-a0664d9c2673 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:42.479720 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6eb84-16c4-4d19-ad27-7c0927c91a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:42.480331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6eb84-16c4-4d19-ad27-7c0927c91a7c admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.480617 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6eb84-16c4-4d19-ad27-7c0927c91a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.480879 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6eb84-16c4-4d19-ad27-7c0927c91a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.482367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6eb84-16c4-4d19-ad27-7c0927c91a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:42.483604 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6eb84-16c4-4d19-ad27-7c0927c91a7c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:42.484557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f074952b3564456abe94dcd0be50531d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:42.485224 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f074952b3564456abe94dcd0be50531d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:42.547276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f074952b3564456abe94dcd0be50531d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:42.566802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1313/2646] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:42 2022] DELETE /identity/v3/users/f074952b3564456abe94dcd0be50531d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:42.567811 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:42.569169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:42.622395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1334/2647] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:42 2022] POST /identity/v3/projects => generated 367 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:42.623773 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6feb4-4dce-4906-9851-dfb1a1855539 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.624055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6feb4-4dce-4906-9851-dfb1a1855539 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.624318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6feb4-4dce-4906-9851-dfb1a1855539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.639376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35ab34-0242-4a49-96eb-ea8e91ff3a56 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:42.642877 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-2bd6feb4-4dce-4906-9851-dfb1a1855539 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:42.647689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2bd6feb4-4dce-4906-9851-dfb1a1855539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:42.698862 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:42.699630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.699909 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.700183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] PATH_INFO: `/v3/projects/609083a2099b4670a5dbf14172eb4dd7` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.701355 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=609083a2099b4670a5dbf14172eb4dd7)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:42.703354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:42.703485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1335/2648] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:42.704183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6feb4-4dce-4906-9851-dfb1a1855539 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:42.705470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2bd6feb4-4dce-4906-9851-dfb1a1855539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:42.718300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6feb4-4dce-4906-9851-dfb1a1855539 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:42.722944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:42.723544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.723814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.724082 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.724685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:42.727419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:42.786299 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:42.833064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1336/2649] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:42 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:42.833887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:42.838410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:42.845447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609083a2099b4670a5dbf14172eb4dd7', 'request_id': 'req-26dc146c-3c6e-4a10-ac60-579ee363afaf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:42.851380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609083a2099b4670a5dbf14172eb4dd7', 'request_id': 'req-26dc146c-3c6e-4a10-ac60-579ee363afaf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:42.900404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca50d-2f8c-49ec-92cb-4a9d660c9e99 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:42.952076 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428534a-c09b-4c79-9263-dea12046e11f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:42.952725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428534a-c09b-4c79-9263-dea12046e11f service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:42.954424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428534a-c09b-4c79-9263-dea12046e11f service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:42.954424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428534a-c09b-4c79-9263-dea12046e11f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:42.986954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609083a2099b4670a5dbf14172eb4dd7', 'request_id': 'req-26dc146c-3c6e-4a10-ac60-579ee363afaf'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:43.000495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] The token cache is being invalidate because project 609083a2099b4670a5dbf14172eb4dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:43.001070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:43.002015 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:43.003552 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1314/2650] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:42 2022] DELETE /identity/v3/projects/609083a2099b4670a5dbf14172eb4dd7 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:43.004472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.005733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:43.027808 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428534a-c09b-4c79-9263-dea12046e11f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.029577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428534a-c09b-4c79-9263-dea12046e11f service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.039899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1337/2651] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:43.040984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387448-557a-49ef-b7ab-68a81fb9d52d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.041262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387448-557a-49ef-b7ab-68a81fb9d52d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.041529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387448-557a-49ef-b7ab-68a81fb9d52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.065405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-be387448-557a-49ef-b7ab-68a81fb9d52d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:43.066667 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-be387448-557a-49ef-b7ab-68a81fb9d52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:43.088900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1338/2652] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:43.088900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625328a-99b5-4ad9-907e-0a0a5fcc43a9 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.088900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625328a-99b5-4ad9-907e-0a0a5fcc43a9 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.088900 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625328a-99b5-4ad9-907e-0a0a5fcc43a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.099731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc146c-3c6e-4a10-ac60-579ee363afaf admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.105367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dca97-63da-43ed-87ce-4afad91630aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.106508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dca97-63da-43ed-87ce-4afad91630aa admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.106806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-3625328a-99b5-4ad9-907e-0a0a5fcc43a9 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:43.107003 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dca97-63da-43ed-87ce-4afad91630aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.107232 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dca97-63da-43ed-87ce-4afad91630aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.108235 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4dca97-63da-43ed-87ce-4afad91630aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.108367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3625328a-99b5-4ad9-907e-0a0a5fcc43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:43.109471 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4dca97-63da-43ed-87ce-4afad91630aa admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.124750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1315/2654] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:43.124961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1339/2654] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:43.125259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dca97-63da-43ed-87ce-4afad91630aa admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.128505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625328a-99b5-4ad9-907e-0a0a5fcc43a9 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.136536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dca97-63da-43ed-87ce-4afad91630aa admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.139323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625328a-99b5-4ad9-907e-0a0a5fcc43a9 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.141660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.141891 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.142133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.142400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.143159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:43.143734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.144755 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07710510-9aef-45cf-b1d8-41081c0139e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.145014 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.145407 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07710510-9aef-45cf-b1d8-41081c0139e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.145806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07710510-9aef-45cf-b1d8-41081c0139e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.146091 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07710510-9aef-45cf-b1d8-41081c0139e0 admin admin] PATH_INFO: `/v3/projects/8700209d0c4e44b581b1276f518db6d2/users/0d67a3c60a0a45eebd34e977959bb26d/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.158568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:43.167836 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1316/2655] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 16:05:43.168621 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07710510-9aef-45cf-b1d8-41081c0139e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8700209d0c4e44b581b1276f518db6d2, user_id=0d67a3c60a0a45eebd34e977959bb26d, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.169283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.170008 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07710510-9aef-45cf-b1d8-41081c0139e0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.180948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c844-b93d-4d1c-a45c-6a9ab0be8d36 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.185855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfdc95-ae51-416d-a996-5e3dd82af5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.186450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfdc95-ae51-416d-a996-5e3dd82af5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.186725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfdc95-ae51-416d-a996-5e3dd82af5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.187030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfdc95-ae51-416d-a996-5e3dd82af5a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.187729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfdc95-ae51-416d-a996-5e3dd82af5a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.189623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfdc95-ae51-416d-a996-5e3dd82af5a2 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.325309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1340/2656] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:43 2022] PUT /identity/v3/projects/8700209d0c4e44b581b1276f518db6d2/users/0d67a3c60a0a45eebd34e977959bb26d/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:43.329494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1317/2657] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:43 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:43.330802 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07710510-9aef-45cf-b1d8-41081c0139e0 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.330993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfdc95-ae51-416d-a996-5e3dd82af5a2 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.402055 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07710510-9aef-45cf-b1d8-41081c0139e0 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.407416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2bb5f-9037-44bc-9676-066a953c6971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.413279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2bb5f-9037-44bc-9676-066a953c6971 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.413279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2bb5f-9037-44bc-9676-066a953c6971 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.413279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2bb5f-9037-44bc-9676-066a953c6971 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.413279 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2bb5f-9037-44bc-9676-066a953c6971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.413714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfdc95-ae51-416d-a996-5e3dd82af5a2 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.413827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2bb5f-9037-44bc-9676-066a953c6971 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.417193 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385497c-fca1-427a-a1a2-2ccd03a99c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.417908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385497c-fca1-427a-a1a2-2ccd03a99c2c admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.418263 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385497c-fca1-427a-a1a2-2ccd03a99c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.418739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385497c-fca1-427a-a1a2-2ccd03a99c2c admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.419666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385497c-fca1-427a-a1a2-2ccd03a99c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.421672 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385497c-fca1-427a-a1a2-2ccd03a99c2c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.423285 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1341/2658] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:43.426647 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2bb5f-9037-44bc-9676-066a953c6971 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.441227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2bb5f-9037-44bc-9676-066a953c6971 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.447439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4732f-a7a8-4d1a-a891-21beaa14abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.448112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4732f-a7a8-4d1a-a891-21beaa14abb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.448390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4732f-a7a8-4d1a-a891-21beaa14abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.448711 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4732f-a7a8-4d1a-a891-21beaa14abb0 admin admin] PATH_INFO: `/v3/projects/8700209d0c4e44b581b1276f518db6d2/users/0d67a3c60a0a45eebd34e977959bb26d/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.450779 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4732f-a7a8-4d1a-a891-21beaa14abb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8700209d0c4e44b581b1276f518db6d2, user_id=0d67a3c60a0a45eebd34e977959bb26d, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.453533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4732f-a7a8-4d1a-a891-21beaa14abb0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.464642 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1342/2659] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:43 2022] PUT /identity/v3/projects/8700209d0c4e44b581b1276f518db6d2/users/0d67a3c60a0a45eebd34e977959bb26d/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:43.490051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b45f-7f4a-4062-a7f7-9600e97b3cdf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.490404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b45f-7f4a-4062-a7f7-9600e97b3cdf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.490792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b45f-7f4a-4062-a7f7-9600e97b3cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.521574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5fc7b45f-7f4a-4062-a7f7-9600e97b3cdf None None] MFA Rules not processed for user `0d67a3c60a0a45eebd34e977959bb26d`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:43.523017 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5fc7b45f-7f4a-4062-a7f7-9600e97b3cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:43.592015 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1343/2660] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:43.604044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b45f-7f4a-4062-a7f7-9600e97b3cdf None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.605367 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5fc7b45f-7f4a-4062-a7f7-9600e97b3cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:43.660042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1318/2661] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:43 2022] POST /identity/v3/users => generated 290 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:43.661545 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b45f-7f4a-4062-a7f7-9600e97b3cdf None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.664687 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385497c-fca1-427a-a1a2-2ccd03a99c2c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.666023 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.666632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.666909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.667207 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.674170 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385497c-fca1-427a-a1a2-2ccd03a99c2c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.676488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.677921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.688408 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1344/2662] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:43.702970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.704656 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:43.747686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724339fb-1a13-4609-89f6-9c5084a36b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.751514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724339fb-1a13-4609-89f6-9c5084a36b17 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.751514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724339fb-1a13-4609-89f6-9c5084a36b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.751514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724339fb-1a13-4609-89f6-9c5084a36b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.751514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724339fb-1a13-4609-89f6-9c5084a36b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.751514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724339fb-1a13-4609-89f6-9c5084a36b17 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.764293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1319/2663] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:43.765310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724339fb-1a13-4609-89f6-9c5084a36b17 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.774880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724339fb-1a13-4609-89f6-9c5084a36b17 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.775174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5c670-e11b-46a9-a189-5edc9e4feefe service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.779467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0e260-ed4e-459a-ba4a-5db559161095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.785980 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.786104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0e260-ed4e-459a-ba4a-5db559161095 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.786202 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.786285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0e260-ed4e-459a-ba4a-5db559161095 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.786379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.786466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0e260-ed4e-459a-ba4a-5db559161095 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.786557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] PATH_INFO: `/v3/projects/6f052a3543484b519aabe91832d6568f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.786648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0e260-ed4e-459a-ba4a-5db559161095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.786738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f052a3543484b519aabe91832d6568f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.786838 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0e260-ed4e-459a-ba4a-5db559161095 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.786960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.793929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1320/2664] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:43.797902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0e260-ed4e-459a-ba4a-5db559161095 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:43.808056 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0e260-ed4e-459a-ba4a-5db559161095 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:43.813296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f0d1a-185d-4c2e-8c06-470bb157685a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:43.813939 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f0d1a-185d-4c2e-8c06-470bb157685a admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.814214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f0d1a-185d-4c2e-8c06-470bb157685a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.814480 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f0d1a-185d-4c2e-8c06-470bb157685a admin admin] PATH_INFO: `/v3/projects/76658496e99b4df7b1f9b4bf584e3d90/users/1d57bd12ed7644ff87c34558a3b8ca7f/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.829392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f0d1a-185d-4c2e-8c06-470bb157685a admin admin] RBAC: Authorizing `identity:create_grant(project_id=76658496e99b4df7b1f9b4bf584e3d90, user_id=1d57bd12ed7644ff87c34558a3b8ca7f, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:43.832075 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f0d1a-185d-4c2e-8c06-470bb157685a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:43.919337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1321/2665] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:43 2022] PUT /identity/v3/projects/76658496e99b4df7b1f9b4bf584e3d90/users/1d57bd12ed7644ff87c34558a3b8ca7f/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:43.927388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f052a3543484b519aabe91832d6568f', 'request_id': 'req-4280e332-a93a-41f1-8f2f-b9193b90b38e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:43.930038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cedac-d9fe-4750-85fc-d99cb3a1263f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:43.930321 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cedac-d9fe-4750-85fc-d99cb3a1263f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:43.930596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cedac-d9fe-4750-85fc-d99cb3a1263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:43.932862 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f052a3543484b519aabe91832d6568f', 'request_id': 'req-4280e332-a93a-41f1-8f2f-b9193b90b38e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:43.958864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-2c9cedac-d9fe-4750-85fc-d99cb3a1263f None None] MFA Rules not processed for user `1d57bd12ed7644ff87c34558a3b8ca7f`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:43.960704 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2c9cedac-d9fe-4750-85fc-d99cb3a1263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:44.003803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f052a3543484b519aabe91832d6568f', 'request_id': 'req-4280e332-a93a-41f1-8f2f-b9193b90b38e'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:44.017846 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] The token cache is being invalidate because project 6f052a3543484b519aabe91832d6568f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:44.018157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:44.018937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:44.020489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1345/2666] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:43 2022] DELETE /identity/v3/projects/6f052a3543484b519aabe91832d6568f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:44.024420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.024420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:44.026900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1322/2667] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:44.027783 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9cedac-d9fe-4750-85fc-d99cb3a1263f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.086117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280e332-a93a-41f1-8f2f-b9193b90b38e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.090086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.090715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9cedac-d9fe-4750-85fc-d99cb3a1263f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.090826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.091192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.091601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.093467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:44.095498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.096129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.096405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.096680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] PATH_INFO: `/v3/users/d7b91e6082c14e899497277c27c25bfe` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.098314 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7b91e6082c14e899497277c27c25bfe)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.100475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:44.157894 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.159412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:44.174858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1346/2668] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:44.174858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.174858 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:44.187620 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3e614-1b4f-42bb-af13-b5cef234a9fb service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.193418 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.193689 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.194070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.194718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.196265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.198211 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:44.322097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7b91e6082c14e899497277c27c25bfe'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:44.322747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7b91e6082c14e899497277c27c25bfe'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:44.335759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7b91e6082c14e899497277c27c25bfe'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:44.336532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7b91e6082c14e899497277c27c25bfe'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:44.338134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1347/2669] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:44 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:44.342228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.343951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:44.403155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7b91e6082c14e899497277c27c25bfe'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:44.409487 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a4f77-5edb-41ec-801f-48c40ed5bb39 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.415296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f028bb-b90b-4b78-9ab4-e54fcf878600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.415907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f028bb-b90b-4b78-9ab4-e54fcf878600 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.416184 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f028bb-b90b-4b78-9ab4-e54fcf878600 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.416476 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f028bb-b90b-4b78-9ab4-e54fcf878600 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.417138 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f028bb-b90b-4b78-9ab4-e54fcf878600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.417743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1323/2670] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:44 2022] DELETE /identity/v3/users/d7b91e6082c14e899497277c27c25bfe => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:44.419284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f028bb-b90b-4b78-9ab4-e54fcf878600 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:44.559167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1348/2671] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:44 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:44.563793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.644710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b99f6-09e3-47de-aa8b-7f1f46ed7164 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.649925 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:44.650474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a4d4f-bcf0-4400-b57d-9828319ad5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.652649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a4d4f-bcf0-4400-b57d-9828319ad5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.652649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a4d4f-bcf0-4400-b57d-9828319ad5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.652649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a4d4f-bcf0-4400-b57d-9828319ad5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.652649 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a4d4f-bcf0-4400-b57d-9828319ad5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.653677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a4d4f-bcf0-4400-b57d-9828319ad5d7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:44.665141 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1324/2672] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:44.669080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f028bb-b90b-4b78-9ab4-e54fcf878600 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.680863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f028bb-b90b-4b78-9ab4-e54fcf878600 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.685645 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.687251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.687251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.687251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.687709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.689314 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:44.701340 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1349/2673] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:44.706052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a4d4f-bcf0-4400-b57d-9828319ad5d7 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.716775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a4d4f-bcf0-4400-b57d-9828319ad5d7 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.721389 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd07b5-2459-4c68-9c89-03fe2ff41c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.722066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd07b5-2459-4c68-9c89-03fe2ff41c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.722336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd07b5-2459-4c68-9c89-03fe2ff41c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.722597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd07b5-2459-4c68-9c89-03fe2ff41c53 admin admin] PATH_INFO: `/v3/projects/17db2a7a0be94061ab3a0391551316a0/users/170c384090944b59a9c9d8c77e48c054/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.738519 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd07b5-2459-4c68-9c89-03fe2ff41c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17db2a7a0be94061ab3a0391551316a0, user_id=170c384090944b59a9c9d8c77e48c054, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.741308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd07b5-2459-4c68-9c89-03fe2ff41c53 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:44.752073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:44.813257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.815253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:44.827368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1325/2674] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:44 2022] PUT /identity/v3/projects/17db2a7a0be94061ab3a0391551316a0/users/170c384090944b59a9c9d8c77e48c054/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:44.840725 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdb8a-af1b-43de-af4e-9e199bb82b51 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.841045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdb8a-af1b-43de-af4e-9e199bb82b51 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.841517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdb8a-af1b-43de-af4e-9e199bb82b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.844483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2867148-bcba-4b84-b8b2-7c37a682609e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.875131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-4bcfdb8a-af1b-43de-af4e-9e199bb82b51 None None] MFA Rules not processed for user `170c384090944b59a9c9d8c77e48c054`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:44.876629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4bcfdb8a-af1b-43de-af4e-9e199bb82b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:44.915080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.915841 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.916326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.916614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] PATH_INFO: `/v3/projects/15c3bb3a8acc4897bbf023e5d5efb496` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.917972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c3bb3a8acc4897bbf023e5d5efb496)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.921362 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:44.954084 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1326/2675] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:44.958497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfdb8a-af1b-43de-af4e-9e199bb82b51 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:44.969985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfdb8a-af1b-43de-af4e-9e199bb82b51 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:44.974887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34407257-5715-45b5-9320-741d81c428f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:44.976129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34407257-5715-45b5-9320-741d81c428f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:44.976398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34407257-5715-45b5-9320-741d81c428f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:44.976668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34407257-5715-45b5-9320-741d81c428f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:44.977287 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34407257-5715-45b5-9320-741d81c428f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:44.979308 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34407257-5715-45b5-9320-741d81c428f0 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.067431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c3bb3a8acc4897bbf023e5d5efb496', 'request_id': 'req-66b7cf64-455f-4574-88d7-a3a3cc910e3d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:45.072941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c3bb3a8acc4897bbf023e5d5efb496', 'request_id': 'req-66b7cf64-455f-4574-88d7-a3a3cc910e3d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:45.234517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1327/2676] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:44 2022] POST /identity/v3/projects => generated 377 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:45.238702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34407257-5715-45b5-9320-741d81c428f0 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.260614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c3bb3a8acc4897bbf023e5d5efb496', 'request_id': 'req-66b7cf64-455f-4574-88d7-a3a3cc910e3d'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:45.272736 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] The token cache is being invalidate because project 15c3bb3a8acc4897bbf023e5d5efb496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:45.273061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:45.273712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-66b7cf64-455f-4574-88d7-a3a3cc910e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:45.275404 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1350/2677] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:44 2022] DELETE /identity/v3/projects/15c3bb3a8acc4897bbf023e5d5efb496 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:45.280259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7365f0a-4cdf-4e8f-a436-a1e562a2a8ef None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.280541 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7365f0a-4cdf-4e8f-a436-a1e562a2a8ef None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.280814 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7365f0a-4cdf-4e8f-a436-a1e562a2a8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.303392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34407257-5715-45b5-9320-741d81c428f0 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.309329 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.310013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.310226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.310512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.311218 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.313158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.317718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-f7365f0a-4cdf-4e8f-a436-a1e562a2a8ef None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:45.318937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f7365f0a-4cdf-4e8f-a436-a1e562a2a8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:45.327445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1351/2678] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:45.335843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa156d7-a5b2-4ddd-b007-b462f70ed828 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.336150 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa156d7-a5b2-4ddd-b007-b462f70ed828 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.336395 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa156d7-a5b2-4ddd-b007-b462f70ed828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.357527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-dfa156d7-a5b2-4ddd-b007-b462f70ed828 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:45.358724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-dfa156d7-a5b2-4ddd-b007-b462f70ed828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:45.366968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1352/2679] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:45.371159 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa156d7-a5b2-4ddd-b007-b462f70ed828 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.378531 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:45.382353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa156d7-a5b2-4ddd-b007-b462f70ed828 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.387284 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.388044 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.388319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.388581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.389703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:45.389703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.391133 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.407330 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1353/2680] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 16:05:45.414317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.414532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1328/2681] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:45 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:45.416972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.418960 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:45.420046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529196d-e1bc-4e70-b953-f26592145bba admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.425047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d4be-7812-4f3a-9311-a0d0d4d0e298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.426072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d4be-7812-4f3a-9311-a0d0d4d0e298 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.426072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d4be-7812-4f3a-9311-a0d0d4d0e298 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.426268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d4be-7812-4f3a-9311-a0d0d4d0e298 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.426741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91d4be-7812-4f3a-9311-a0d0d4d0e298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.429402 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91d4be-7812-4f3a-9311-a0d0d4d0e298 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.432395 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6206622-1173-45dd-a67d-c53b4037d392 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.437281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37c230-409b-42f7-89b4-f53ebf328ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.438111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c230-409b-42f7-89b4-f53ebf328ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.438242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c230-409b-42f7-89b4-f53ebf328ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.438421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c230-409b-42f7-89b4-f53ebf328ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.439301 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37c230-409b-42f7-89b4-f53ebf328ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.441161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37c230-409b-42f7-89b4-f53ebf328ac9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.452370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1329/2682] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:45.456627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37c230-409b-42f7-89b4-f53ebf328ac9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.467096 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37c230-409b-42f7-89b4-f53ebf328ac9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.471800 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c90132-7626-4aee-b233-92318924714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.472456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c90132-7626-4aee-b233-92318924714a admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.472735 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c90132-7626-4aee-b233-92318924714a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.473010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c90132-7626-4aee-b233-92318924714a admin admin] PATH_INFO: `/v3/projects/0890e892d7814ca88d28264d2308b1a2/users/96c679c2c56046ceb54a8ffd23bc4718/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.488245 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c90132-7626-4aee-b233-92318924714a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0890e892d7814ca88d28264d2308b1a2, user_id=96c679c2c56046ceb54a8ffd23bc4718, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.490850 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c90132-7626-4aee-b233-92318924714a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.524716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1354/2683] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:45 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:45.528759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d4be-7812-4f3a-9311-a0d0d4d0e298 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.540700 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:45.581063 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1330/2684] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:45 2022] PUT /identity/v3/projects/0890e892d7814ca88d28264d2308b1a2/users/96c679c2c56046ceb54a8ffd23bc4718/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:45.585250 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c90132-7626-4aee-b233-92318924714a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.593205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d4be-7812-4f3a-9311-a0d0d4d0e298 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.604213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:45.648379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c90132-7626-4aee-b233-92318924714a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.652423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ca238-6422-46b3-8081-bbdffe3fe438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.653100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca238-6422-46b3-8081-bbdffe3fe438 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.653226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429301d-3aad-4775-99b8-8029a6077a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.653359 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca238-6422-46b3-8081-bbdffe3fe438 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.653635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca238-6422-46b3-8081-bbdffe3fe438 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.653749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429301d-3aad-4775-99b8-8029a6077a5c admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.654011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429301d-3aad-4775-99b8-8029a6077a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.654255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ca238-6422-46b3-8081-bbdffe3fe438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.654356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429301d-3aad-4775-99b8-8029a6077a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.654908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429301d-3aad-4775-99b8-8029a6077a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.656405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ca238-6422-46b3-8081-bbdffe3fe438 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.659721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429301d-3aad-4775-99b8-8029a6077a5c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.669706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1331/2685] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:45.670727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429301d-3aad-4775-99b8-8029a6077a5c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.680887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429301d-3aad-4775-99b8-8029a6077a5c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.685611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89010c-96b5-4f0d-9845-849502e77f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.686243 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89010c-96b5-4f0d-9845-849502e77f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.686522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89010c-96b5-4f0d-9845-849502e77f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.686795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89010c-96b5-4f0d-9845-849502e77f20 admin admin] PATH_INFO: `/v3/projects/0890e892d7814ca88d28264d2308b1a2/users/96c679c2c56046ceb54a8ffd23bc4718/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.688761 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89010c-96b5-4f0d-9845-849502e77f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0890e892d7814ca88d28264d2308b1a2, user_id=96c679c2c56046ceb54a8ffd23bc4718, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.691695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89010c-96b5-4f0d-9845-849502e77f20 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.702284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1332/2686] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:45 2022] PUT /identity/v3/projects/0890e892d7814ca88d28264d2308b1a2/users/96c679c2c56046ceb54a8ffd23bc4718/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:45.712544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0b778-3117-4a3f-8116-8b5a69d0a2ae None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.712829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0b778-3117-4a3f-8116-8b5a69d0a2ae None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.713120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0b778-3117-4a3f-8116-8b5a69d0a2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.740759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6ed0b778-3117-4a3f-8116-8b5a69d0a2ae None None] MFA Rules not processed for user `96c679c2c56046ceb54a8ffd23bc4718`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:45.742616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6ed0b778-3117-4a3f-8116-8b5a69d0a2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:45.763392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1355/2687] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:45 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:45.767240 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ca238-6422-46b3-8081-bbdffe3fe438 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.777363 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ca238-6422-46b3-8081-bbdffe3fe438 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.793634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:45.813005 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1333/2688] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:45.826209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0b778-3117-4a3f-8116-8b5a69d0a2ae None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.827698 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6ed0b778-3117-4a3f-8116-8b5a69d0a2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:45.837767 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd290da-1de3-492a-b1b5-a9b0e4f83b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.838345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd290da-1de3-492a-b1b5-a9b0e4f83b07 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.838628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd290da-1de3-492a-b1b5-a9b0e4f83b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.838876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd290da-1de3-492a-b1b5-a9b0e4f83b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.839576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd290da-1de3-492a-b1b5-a9b0e4f83b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.841148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd290da-1de3-492a-b1b5-a9b0e4f83b07 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.851666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1356/2689] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:45.855574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd290da-1de3-492a-b1b5-a9b0e4f83b07 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.865609 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd290da-1de3-492a-b1b5-a9b0e4f83b07 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.871053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c8623-87d3-4053-8b95-a9321a389c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.871644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c8623-87d3-4053-8b95-a9321a389c99 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.872309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c8623-87d3-4053-8b95-a9321a389c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.872309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c8623-87d3-4053-8b95-a9321a389c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.872893 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c8623-87d3-4053-8b95-a9321a389c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.874460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c8623-87d3-4053-8b95-a9321a389c99 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.886864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1357/2690] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:45.888661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0b778-3117-4a3f-8116-8b5a69d0a2ae None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.891142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c8623-87d3-4053-8b95-a9321a389c99 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:45.892844 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae779f5-a681-4bdb-8306-b791d60179f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.893656 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae779f5-a681-4bdb-8306-b791d60179f5 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.894114 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae779f5-a681-4bdb-8306-b791d60179f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.894554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae779f5-a681-4bdb-8306-b791d60179f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.902066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c8623-87d3-4053-8b95-a9321a389c99 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:45.906575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae779f5-a681-4bdb-8306-b791d60179f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.907024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175a297-7190-42cb-9357-af6deac7cc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:45.907679 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175a297-7190-42cb-9357-af6deac7cc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:45.907957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175a297-7190-42cb-9357-af6deac7cc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:45.908078 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae779f5-a681-4bdb-8306-b791d60179f5 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:45.908388 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175a297-7190-42cb-9357-af6deac7cc9b admin admin] PATH_INFO: `/v3/projects/32568f74fbfb4ee98236a276a8acfaeb/users/6b7e5ad0c91d41cf8e118934dcd9e35f/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:45.918703 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1334/2691] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:45.932079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175a297-7190-42cb-9357-af6deac7cc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=32568f74fbfb4ee98236a276a8acfaeb, user_id=6b7e5ad0c91d41cf8e118934dcd9e35f, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:45.934883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175a297-7190-42cb-9357-af6deac7cc9b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:46.088049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae779f5-a681-4bdb-8306-b791d60179f5 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:46.092881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:46.096703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1358/2692] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:45 2022] PUT /identity/v3/projects/32568f74fbfb4ee98236a276a8acfaeb/users/6b7e5ad0c91d41cf8e118934dcd9e35f/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:46.101734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae779f5-a681-4bdb-8306-b791d60179f5 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:46.108014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdcef3-2f27-430f-8120-537a8937345f None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:46.108295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdcef3-2f27-430f-8120-537a8937345f None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:46.108560 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdcef3-2f27-430f-8120-537a8937345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:46.136939 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-9fbdcef3-2f27-430f-8120-537a8937345f None None] MFA Rules not processed for user `6b7e5ad0c91d41cf8e118934dcd9e35f`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:46.138118 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-9fbdcef3-2f27-430f-8120-537a8937345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:46.152291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:46.152916 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:46.153189 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:46.153460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:46.155694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:46.203056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1359/2693] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:46.207611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdcef3-2f27-430f-8120-537a8937345f None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:46.219466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:46.220902 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:46.229865 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1335/2694] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:46.278978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdcef3-2f27-430f-8120-537a8937345f None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:46.284095 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c03c5-861b-42c0-a30c-25ffb8ba649f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:46.285122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c03c5-861b-42c0-a30c-25ffb8ba649f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:46.285397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c03c5-861b-42c0-a30c-25ffb8ba649f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:46.285661 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c03c5-861b-42c0-a30c-25ffb8ba649f admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:46.286278 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c03c5-861b-42c0-a30c-25ffb8ba649f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:46.288934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c03c5-861b-42c0-a30c-25ffb8ba649f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:46.426143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:46.437817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1360/2695] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:46 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:46.438760 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:46.450950 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac68861-74cd-48ae-b544-7e1133d33ac0 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:46.519718 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ee987-b1ed-43e6-b81b-0a537cd6007e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:46.520439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ee987-b1ed-43e6-b81b-0a537cd6007e admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:46.520717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ee987-b1ed-43e6-b81b-0a537cd6007e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:46.520986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ee987-b1ed-43e6-b81b-0a537cd6007e admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:46.521716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ee987-b1ed-43e6-b81b-0a537cd6007e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:46.524022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ee987-b1ed-43e6-b81b-0a537cd6007e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:46.591884 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:46.671984 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1336/2696] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:46 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:46.683337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c03c5-861b-42c0-a30c-25ffb8ba649f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:46.693842 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c03c5-861b-42c0-a30c-25ffb8ba649f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:46.699333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dd522-e265-4ddf-b455-e1740c1adbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:46.699942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd522-e265-4ddf-b455-e1740c1adbdf admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:46.700204 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd522-e265-4ddf-b455-e1740c1adbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:46.700474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dd522-e265-4ddf-b455-e1740c1adbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:46.701151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dd522-e265-4ddf-b455-e1740c1adbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:46.702737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dd522-e265-4ddf-b455-e1740c1adbdf admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:46.713480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1361/2697] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:46.717459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ee987-b1ed-43e6-b81b-0a537cd6007e admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:46.727765 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ee987-b1ed-43e6-b81b-0a537cd6007e admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:46.732436 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038440a-8431-4a6e-9864-c926d3f6586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:46.733407 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038440a-8431-4a6e-9864-c926d3f6586a admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:46.733696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038440a-8431-4a6e-9864-c926d3f6586a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:46.743714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038440a-8431-4a6e-9864-c926d3f6586a admin admin] PATH_INFO: `/v3/projects/0ece52be5abb4b60bb0cfb422f9fc374/users/97a363df6a594eb9aa3fe98f504e0806/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:46.754336 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038440a-8431-4a6e-9864-c926d3f6586a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ece52be5abb4b60bb0cfb422f9fc374, user_id=97a363df6a594eb9aa3fe98f504e0806, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:46.757714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038440a-8431-4a6e-9864-c926d3f6586a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:46.890655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1337/2698] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:46 2022] PUT /identity/v3/projects/0ece52be5abb4b60bb0cfb422f9fc374/users/97a363df6a594eb9aa3fe98f504e0806/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:46.894518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dd522-e265-4ddf-b455-e1740c1adbdf admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:46.904715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dd522-e265-4ddf-b455-e1740c1adbdf admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:46.962093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bc5a5-0b28-4a0c-a452-8424c92dc6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:46.962803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bc5a5-0b28-4a0c-a452-8424c92dc6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:46.963114 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bc5a5-0b28-4a0c-a452-8424c92dc6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:46.963382 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bc5a5-0b28-4a0c-a452-8424c92dc6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:46.964019 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bc5a5-0b28-4a0c-a452-8424c92dc6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:46.965594 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bc5a5-0b28-4a0c-a452-8424c92dc6e0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:46.976760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1362/2699] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:46.981073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038440a-8431-4a6e-9864-c926d3f6586a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:46.993095 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038440a-8431-4a6e-9864-c926d3f6586a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:46.998085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:46.998763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:46.999073 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:46.999378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] PATH_INFO: `/v3/projects/0ece52be5abb4b60bb0cfb422f9fc374/users/97a363df6a594eb9aa3fe98f504e0806/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:47.001555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ece52be5abb4b60bb0cfb422f9fc374, user_id=97a363df6a594eb9aa3fe98f504e0806, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:47.004036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:47.019968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1338/2700] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:46 2022] PUT /identity/v3/projects/0ece52be5abb4b60bb0cfb422f9fc374/users/97a363df6a594eb9aa3fe98f504e0806/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:47.030577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900e082-b55d-4d91-bd7b-eae595d7705d None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:47.030794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900e082-b55d-4d91-bd7b-eae595d7705d None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:47.031101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900e082-b55d-4d91-bd7b-eae595d7705d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:47.060551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1900e082-b55d-4d91-bd7b-eae595d7705d None None] MFA Rules not processed for user `97a363df6a594eb9aa3fe98f504e0806`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:47.061764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1900e082-b55d-4d91-bd7b-eae595d7705d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:47.140712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1363/2701] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:47.148923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:47.151544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:47.212257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a756fd0-cd7b-460c-a0ba-8efdc40bc7ae admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:47.216161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1348f57-a158-43aa-9892-171158d142f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:47.216773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348f57-a158-43aa-9892-171158d142f7 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:47.217047 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348f57-a158-43aa-9892-171158d142f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:47.217315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348f57-a158-43aa-9892-171158d142f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:47.227660 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1348f57-a158-43aa-9892-171158d142f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:47.229175 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1348f57-a158-43aa-9892-171158d142f7 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:47.239439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1339/2702] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:47.309946 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900e082-b55d-4d91-bd7b-eae595d7705d None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:47.380936 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:47.383838 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900e082-b55d-4d91-bd7b-eae595d7705d None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:47.388763 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:47.389517 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:47.389664 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:47.389961 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] PATH_INFO: `/v3/users/6b7e5ad0c91d41cf8e118934dcd9e35f` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:47.391128 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7e5ad0c91d41cf8e118934dcd9e35f)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:47.393036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:47.602573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7e5ad0c91d41cf8e118934dcd9e35f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:47.603431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7e5ad0c91d41cf8e118934dcd9e35f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:47.615521 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7e5ad0c91d41cf8e118934dcd9e35f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:47.729390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7e5ad0c91d41cf8e118934dcd9e35f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:47.736966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7e5ad0c91d41cf8e118934dcd9e35f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:47.743608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1364/2703] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:47 2022] DELETE /identity/v3/users/6b7e5ad0c91d41cf8e118934dcd9e35f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:47.748501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761dab-76e6-4339-947d-ca6351dc7c64 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:47.748998 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761dab-76e6-4339-947d-ca6351dc7c64 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:47.749119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761dab-76e6-4339-947d-ca6351dc7c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:47.768353 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-eb761dab-76e6-4339-947d-ca6351dc7c64 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:47.769553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-eb761dab-76e6-4339-947d-ca6351dc7c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:47.836630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1340/2704] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:47.847390 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:47.852951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:47.884731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:47.916356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a1d26-839c-4bea-a6b4-427c5ac84dad admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:47.920444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7525b-8993-4b92-b196-1290b814cb82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:47.921061 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7525b-8993-4b92-b196-1290b814cb82 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:47.921331 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7525b-8993-4b92-b196-1290b814cb82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:47.921593 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7525b-8993-4b92-b196-1290b814cb82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:47.931485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7525b-8993-4b92-b196-1290b814cb82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:47.932943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7525b-8993-4b92-b196-1290b814cb82 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:47.943881 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1365/2705] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:48.106670 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:48.395829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb761dab-76e6-4339-947d-ca6351dc7c64 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:48.397457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-eb761dab-76e6-4339-947d-ca6351dc7c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:48.419006 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb761dab-76e6-4339-947d-ca6351dc7c64 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:48.423856 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:48.424498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:48.424773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:48.425258 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] PATH_INFO: `/v3/projects/32568f74fbfb4ee98236a276a8acfaeb` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:48.426176 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32568f74fbfb4ee98236a276a8acfaeb)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:48.428157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:48.643257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:48.688706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32568f74fbfb4ee98236a276a8acfaeb', 'request_id': 'req-7edeb118-1e21-45ab-b8ef-357256671c21'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:48.705199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32568f74fbfb4ee98236a276a8acfaeb', 'request_id': 'req-7edeb118-1e21-45ab-b8ef-357256671c21'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:48.780712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32568f74fbfb4ee98236a276a8acfaeb', 'request_id': 'req-7edeb118-1e21-45ab-b8ef-357256671c21'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:48.796316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] The token cache is being invalidate because project 32568f74fbfb4ee98236a276a8acfaeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:48.796961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:48.798197 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:48.800826 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1341/2706] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:48 2022] DELETE /identity/v3/projects/32568f74fbfb4ee98236a276a8acfaeb => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:48.808532 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7525b-8993-4b92-b196-1290b814cb82 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:48.824901 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7525b-8993-4b92-b196-1290b814cb82 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:48.955807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:48.956558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:48.956873 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:48.957228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] PATH_INFO: `/v3/users/97a363df6a594eb9aa3fe98f504e0806` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:48.958378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a363df6a594eb9aa3fe98f504e0806)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:48.960436 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:49.412271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a363df6a594eb9aa3fe98f504e0806'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:49.413042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a363df6a594eb9aa3fe98f504e0806'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:49.426021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a363df6a594eb9aa3fe98f504e0806'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:49.580210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a363df6a594eb9aa3fe98f504e0806'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:49.588516 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-29ffb6c3-2e39-423a-a383-079ed7c76e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a363df6a594eb9aa3fe98f504e0806'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:49.594653 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1366/2707] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:48 2022] DELETE /identity/v3/users/97a363df6a594eb9aa3fe98f504e0806 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:49.621718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:50.106788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:50.108350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:50.130596 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edeb118-1e21-45ab-b8ef-357256671c21 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:50.173249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:50.203169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:50.204221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:50.204544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:50.204757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] PATH_INFO: `/v3/projects/0ece52be5abb4b60bb0cfb422f9fc374` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:50.205913 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ece52be5abb4b60bb0cfb422f9fc374)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:50.207886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:50.363532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ece52be5abb4b60bb0cfb422f9fc374', 'request_id': 'req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:50.369288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ece52be5abb4b60bb0cfb422f9fc374', 'request_id': 'req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:50.454373 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ece52be5abb4b60bb0cfb422f9fc374', 'request_id': 'req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:50.467790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] The token cache is being invalidate because project 0ece52be5abb4b60bb0cfb422f9fc374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:50.468082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:50.471162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-0c6489f4-3a45-40d6-b71c-d8fc0c56676f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:50.471162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1342/2708] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:50 2022] DELETE /identity/v3/projects/0ece52be5abb4b60bb0cfb422f9fc374 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:50.475166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec7c90-0260-44ec-95b8-77d79f52af5c None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:50.475760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec7c90-0260-44ec-95b8-77d79f52af5c None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:50.475760 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec7c90-0260-44ec-95b8-77d79f52af5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:50.495509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-8fec7c90-0260-44ec-95b8-77d79f52af5c None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:50.496715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-8fec7c90-0260-44ec-95b8-77d79f52af5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:50.555859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1367/2709] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:50.565554 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7f5fa-6966-4431-8837-d222b4d24b40 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:50.565830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7f5fa-6966-4431-8837-d222b4d24b40 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:50.566115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7f5fa-6966-4431-8837-d222b4d24b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:50.587544 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f4d7f5fa-6966-4431-8837-d222b4d24b40 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:50.588765 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f4d7f5fa-6966-4431-8837-d222b4d24b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:50.596933 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1343/2710] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:50.601466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec7c90-0260-44ec-95b8-77d79f52af5c None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:50.611568 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec7c90-0260-44ec-95b8-77d79f52af5c None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:50.616274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:50.617306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:50.617576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:50.617834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:50.618630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:50.618968 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:50.620587 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:50.633176 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1368/2711] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:05:50.639183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7f5fa-6966-4431-8837-d222b4d24b40 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:50.648889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7f5fa-6966-4431-8837-d222b4d24b40 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:50.653716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74d7d0-0744-4aeb-8755-495833b576a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:50.654313 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74d7d0-0744-4aeb-8755-495833b576a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:50.654591 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74d7d0-0744-4aeb-8755-495833b576a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:50.654863 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74d7d0-0744-4aeb-8755-495833b576a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:50.655531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74d7d0-0744-4aeb-8755-495833b576a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:50.657450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74d7d0-0744-4aeb-8755-495833b576a9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:50.770734 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1344/2712] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:50 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:50.774757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:50.784962 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464555a4-f8cb-419e-9405-f31d27590cf3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:50.843827 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d206390-76d8-421b-b237-eb4f1002a798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:50.844427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206390-76d8-421b-b237-eb4f1002a798 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:50.844699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206390-76d8-421b-b237-eb4f1002a798 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:50.844965 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206390-76d8-421b-b237-eb4f1002a798 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:50.845574 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d206390-76d8-421b-b237-eb4f1002a798 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:50.847508 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d206390-76d8-421b-b237-eb4f1002a798 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:50.981295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1369/2713] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:50 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:50.985462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74d7d0-0744-4aeb-8755-495833b576a9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:50.996982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74d7d0-0744-4aeb-8755-495833b576a9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:51.002370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fb06a-b60a-45d0-990b-4f4ff2750346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:51.003031 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb06a-b60a-45d0-990b-4f4ff2750346 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:51.003307 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb06a-b60a-45d0-990b-4f4ff2750346 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:51.003580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb06a-b60a-45d0-990b-4f4ff2750346 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:51.004221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fb06a-b60a-45d0-990b-4f4ff2750346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:51.005821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0fb06a-b60a-45d0-990b-4f4ff2750346 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:51.016266 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1345/2714] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:51.020364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d206390-76d8-421b-b237-eb4f1002a798 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:51.030717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d206390-76d8-421b-b237-eb4f1002a798 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:51.035592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7253b-8e8c-41b2-b32a-c172ab58e5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:51.036183 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7253b-8e8c-41b2-b32a-c172ab58e5eb admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:51.036456 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7253b-8e8c-41b2-b32a-c172ab58e5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:51.036724 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7253b-8e8c-41b2-b32a-c172ab58e5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:51.037357 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7253b-8e8c-41b2-b32a-c172ab58e5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:51.039654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7253b-8e8c-41b2-b32a-c172ab58e5eb admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:51.050375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1370/2715] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:51.054526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fb06a-b60a-45d0-990b-4f4ff2750346 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:51.066339 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fb06a-b60a-45d0-990b-4f4ff2750346 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:51.072282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:51.073946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:51.074230 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:51.074586 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] PATH_INFO: `/v3/projects/ae1d1f686c5147518ba5b391af2f1a83/users/5c62754c3bc9492486d44f7b3a3add9b/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:51.219439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1d1f686c5147518ba5b391af2f1a83, user_id=5c62754c3bc9492486d44f7b3a3add9b, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:51.222121 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:51.307250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:51.380977 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1346/2716] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:51 2022] PUT /identity/v3/projects/ae1d1f686c5147518ba5b391af2f1a83/users/5c62754c3bc9492486d44f7b3a3add9b/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:51.391533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6b339-cad7-4dff-8986-00061b4005c6 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:51.391817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6b339-cad7-4dff-8986-00061b4005c6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:51.392087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6b339-cad7-4dff-8986-00061b4005c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:51.424953 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-80b6b339-cad7-4dff-8986-00061b4005c6 None None] MFA Rules not processed for user `5c62754c3bc9492486d44f7b3a3add9b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:51.426509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-80b6b339-cad7-4dff-8986-00061b4005c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:51.431375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:51.433795 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:51.529749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:51.534462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fbf98-3c22-4a20-bca1-7827ebc86836 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:51.542611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:51.546828 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:51.547158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1371/2717] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:51.547597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:51.547597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] PATH_INFO: `/v3/users/1d57bd12ed7644ff87c34558a3b8ca7f` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:51.547597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d57bd12ed7644ff87c34558a3b8ca7f)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:51.547597 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:51.548356 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6b339-cad7-4dff-8986-00061b4005c6 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:51.563929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6b339-cad7-4dff-8986-00061b4005c6 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:51.566200 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:51.567026 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:51.567474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:51.567741 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:51.568415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:51.573942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:51.747010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1372/2718] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:51 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:51.748036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:51.749624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:51.805957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbff56e-653b-451e-aedd-593642d9400e admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:51.809754 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:51.810394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:51.810666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:51.810951 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:51.813984 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:51.834594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d57bd12ed7644ff87c34558a3b8ca7f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:51.835402 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d57bd12ed7644ff87c34558a3b8ca7f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:51.848215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d57bd12ed7644ff87c34558a3b8ca7f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:51.848881 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d57bd12ed7644ff87c34558a3b8ca7f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:51.884934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:51.886452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:51.896354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1373/2719] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:51.903016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:51.923163 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d57bd12ed7644ff87c34558a3b8ca7f'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:51.941986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1347/2720] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:51 2022] DELETE /identity/v3/users/1d57bd12ed7644ff87c34558a3b8ca7f => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:51.941986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:51.941986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:51.972766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92747e21-0b6c-4097-ad7c-cd3a67845f0f service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.013511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1ba56-8854-4a28-adcf-cf73b4029657 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.018150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.018771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.019079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.019351 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] PATH_INFO: `/v3/users/a78e9f89d5a5490ba236545b8d95a851` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.020484 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a78e9f89d5a5490ba236545b8d95a851)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.022410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.031666 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c638d1-d83a-43d1-9241-fe2db2f5b180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.032290 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c638d1-d83a-43d1-9241-fe2db2f5b180 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.032566 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c638d1-d83a-43d1-9241-fe2db2f5b180 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.032836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c638d1-d83a-43d1-9241-fe2db2f5b180 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.033464 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c638d1-d83a-43d1-9241-fe2db2f5b180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.036103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c638d1-d83a-43d1-9241-fe2db2f5b180 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.141048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:52.232867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1374/2721] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:51 2022] POST /identity/v3/users => generated 298 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:52.234256 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284355-4e9b-4b02-a2d9-cae2b8828064 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.234635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284355-4e9b-4b02-a2d9-cae2b8828064 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.235033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284355-4e9b-4b02-a2d9-cae2b8828064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.255339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-47284355-4e9b-4b02-a2d9-cae2b8828064 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:52.256553 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-47284355-4e9b-4b02-a2d9-cae2b8828064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:52.265153 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1375/2722] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:52.265950 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284355-4e9b-4b02-a2d9-cae2b8828064 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:52.267213 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-47284355-4e9b-4b02-a2d9-cae2b8828064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:52.279580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284355-4e9b-4b02-a2d9-cae2b8828064 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.284375 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.284981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.285462 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.285734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.286379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.288027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.298637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1376/2723] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:52.298637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:52.303444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:52.366527 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab69668-6a7d-415c-919c-da40cfffe18c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.367900 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a78e9f89d5a5490ba236545b8d95a851'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:52.369435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a78e9f89d5a5490ba236545b8d95a851'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:52.372194 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cb4e2-ad36-47b9-b95d-91acfd605183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.372942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb4e2-ad36-47b9-b95d-91acfd605183 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.373322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb4e2-ad36-47b9-b95d-91acfd605183 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.373819 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb4e2-ad36-47b9-b95d-91acfd605183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.381937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a78e9f89d5a5490ba236545b8d95a851'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:52.382632 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a78e9f89d5a5490ba236545b8d95a851'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:52.384949 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cb4e2-ad36-47b9-b95d-91acfd605183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.386544 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cb4e2-ad36-47b9-b95d-91acfd605183 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.397253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1377/2724] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:52.398117 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cb4e2-ad36-47b9-b95d-91acfd605183 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:52.407702 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cb4e2-ad36-47b9-b95d-91acfd605183 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.412578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.413297 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.413643 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.413978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] PATH_INFO: `/v3/projects/4fdb2e3c07aa475a952ecac0d31e4994/users/f267bf925f7b40d1be7d5f5c14219096/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.429070 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fdb2e3c07aa475a952ecac0d31e4994, user_id=f267bf925f7b40d1be7d5f5c14219096, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.432353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.483231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9c138e7-7b2e-4b64-8851-29a029c24b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a78e9f89d5a5490ba236545b8d95a851'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:52.500930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:52.500930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1348/2725] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:51 2022] DELETE /identity/v3/users/a78e9f89d5a5490ba236545b8d95a851 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:52.502815 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99842f7-e318-4893-886b-53a459ab6edb None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.503397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99842f7-e318-4893-886b-53a459ab6edb None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.503690 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99842f7-e318-4893-886b-53a459ab6edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.523642 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-e99842f7-e318-4893-886b-53a459ab6edb None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:52.525452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1378/2726] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:52 2022] PUT /identity/v3/projects/4fdb2e3c07aa475a952ecac0d31e4994/users/f267bf925f7b40d1be7d5f5c14219096/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:52.528356 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e99842f7-e318-4893-886b-53a459ab6edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:52.529519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:52.531333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:52.592587 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1349/2727] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:52.604092 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525bd98-39dc-46e6-9f72-8ce614d68fbc admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.610206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511cc64-6dd3-4d1c-b3d0-b3b7dd126cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.610631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511cc64-6dd3-4d1c-b3d0-b3b7dd126cae admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.610978 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511cc64-6dd3-4d1c-b3d0-b3b7dd126cae admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.611210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511cc64-6dd3-4d1c-b3d0-b3b7dd126cae admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.612500 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511cc64-6dd3-4d1c-b3d0-b3b7dd126cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.613445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511cc64-6dd3-4d1c-b3d0-b3b7dd126cae admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.615898 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99842f7-e318-4893-886b-53a459ab6edb None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:52.618061 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e99842f7-e318-4893-886b-53a459ab6edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:52.628504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1379/2728] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:52.629392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511cc64-6dd3-4d1c-b3d0-b3b7dd126cae admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:52.641010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511cc64-6dd3-4d1c-b3d0-b3b7dd126cae admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.646592 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfea1a-84ba-4279-bb85-beabc949f8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.647296 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfea1a-84ba-4279-bb85-beabc949f8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.647572 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfea1a-84ba-4279-bb85-beabc949f8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.647834 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfea1a-84ba-4279-bb85-beabc949f8a4 admin admin] PATH_INFO: `/v3/projects/4fdb2e3c07aa475a952ecac0d31e4994/users/f267bf925f7b40d1be7d5f5c14219096/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.650806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfea1a-84ba-4279-bb85-beabc949f8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fdb2e3c07aa475a952ecac0d31e4994, user_id=f267bf925f7b40d1be7d5f5c14219096, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.653675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfea1a-84ba-4279-bb85-beabc949f8a4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.668251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1380/2729] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:52 2022] PUT /identity/v3/projects/4fdb2e3c07aa475a952ecac0d31e4994/users/f267bf925f7b40d1be7d5f5c14219096/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:52.685252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99842f7-e318-4893-886b-53a459ab6edb None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.687018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4782a-a395-4105-ae72-df856368dfcf None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.687295 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4782a-a395-4105-ae72-df856368dfcf None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.687704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4782a-a395-4105-ae72-df856368dfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.698944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:52.723991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-7bd4782a-a395-4105-ae72-df856368dfcf None None] MFA Rules not processed for user `f267bf925f7b40d1be7d5f5c14219096`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:52.725218 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7bd4782a-a395-4105-ae72-df856368dfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:52.766617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.767552 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.767929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.768317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.812122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1381/2730] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:52.824252 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4782a-a395-4105-ae72-df856368dfcf None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:52.825687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7bd4782a-a395-4105-ae72-df856368dfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:52.858273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.859804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.874786 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1350/2731] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:52.895738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4782a-a395-4105-ae72-df856368dfcf None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:52.900742 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd4727-5702-4173-ab0a-a9e69649519a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:52.901365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4727-5702-4173-ab0a-a9e69649519a service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:52.901725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4727-5702-4173-ab0a-a9e69649519a service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:52.901945 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd4727-5702-4173-ab0a-a9e69649519a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:52.916440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd4727-5702-4173-ab0a-a9e69649519a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:52.917892 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd4727-5702-4173-ab0a-a9e69649519a service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:52.928033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1382/2732] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:52.969630 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:52.971665 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:53.009612 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d3ee-e36e-4c56-94db-1feb7441b84e service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:53.039861 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:53.041029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:53.041262 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:53.041584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] PATH_INFO: `/v3/projects/76658496e99b4df7b1f9b4bf584e3d90` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:53.044155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76658496e99b4df7b1f9b4bf584e3d90)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:53.046162 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:53.057540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd4727-5702-4173-ab0a-a9e69649519a service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:53.066787 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd4727-5702-4173-ab0a-a9e69649519a service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:53.070505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:53.071618 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:53.072072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:53.072341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:53.074096 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:53.136732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:53.138098 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:53.148307 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1383/2733] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:53.287805 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76658496e99b4df7b1f9b4bf584e3d90', 'request_id': 'req-db35af91-c418-4384-a110-93d94daeec74'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.289941 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:53.292710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76658496e99b4df7b1f9b4bf584e3d90', 'request_id': 'req-db35af91-c418-4384-a110-93d94daeec74'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.352584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:53.409173 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76658496e99b4df7b1f9b4bf584e3d90', 'request_id': 'req-db35af91-c418-4384-a110-93d94daeec74'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.421990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] The token cache is being invalidate because project 76658496e99b4df7b1f9b4bf584e3d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:53.422276 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.422937 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.424430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1351/2734] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:52 2022] DELETE /identity/v3/projects/76658496e99b4df7b1f9b4bf584e3d90 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:53.428328 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:53.443288 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df56a72-77ae-461d-8a4f-01b1f244aa47 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:53.455636 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:53.466255 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:53.512716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:53.513364 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:53.513641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:53.513912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] PATH_INFO: `/v3/users/170c384090944b59a9c9d8c77e48c054` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:53.517042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170c384090944b59a9c9d8c77e48c054)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:53.519144 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:53.553304 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35af91-c418-4384-a110-93d94daeec74 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:53.558768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:53.559431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:53.559708 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:53.559975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] PATH_INFO: `/v3/projects/ef490e4ac5d0483ab0f446e5d46355d1` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:53.561369 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef490e4ac5d0483ab0f446e5d46355d1)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:53.563239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:53.733513 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170c384090944b59a9c9d8c77e48c054'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.734466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170c384090944b59a9c9d8c77e48c054'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.735129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef490e4ac5d0483ab0f446e5d46355d1', 'request_id': 'req-b9d9cc5f-536d-44b6-aa09-97a8cd539148'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.741160 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef490e4ac5d0483ab0f446e5d46355d1', 'request_id': 'req-b9d9cc5f-536d-44b6-aa09-97a8cd539148'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.747195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170c384090944b59a9c9d8c77e48c054'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.808767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef490e4ac5d0483ab0f446e5d46355d1', 'request_id': 'req-b9d9cc5f-536d-44b6-aa09-97a8cd539148'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.823594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] The token cache is being invalidate because project ef490e4ac5d0483ab0f446e5d46355d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:53.823932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.824623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.826136 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1352/2735] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:53 2022] DELETE /identity/v3/projects/ef490e4ac5d0483ab0f446e5d46355d1 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:53.829965 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:53.831425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:53.853982 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:53.889057 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170c384090944b59a9c9d8c77e48c054'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.898195 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170c384090944b59a9c9d8c77e48c054'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:53.902216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9cc5f-536d-44b6-aa09-97a8cd539148 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:53.905598 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1384/2736] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:53 2022] DELETE /identity/v3/users/170c384090944b59a9c9d8c77e48c054 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:53.907461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:53.908157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:53.911304 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:53.911701 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] PATH_INFO: `/v3/users/0d67a3c60a0a45eebd34e977959bb26d` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:53.913017 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d67a3c60a0a45eebd34e977959bb26d)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:53.916165 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:54.208610 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d67a3c60a0a45eebd34e977959bb26d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:54.209743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d67a3c60a0a45eebd34e977959bb26d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:54.223567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d67a3c60a0a45eebd34e977959bb26d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:54.224885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d67a3c60a0a45eebd34e977959bb26d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:54.330016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d67a3c60a0a45eebd34e977959bb26d'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:54.351010 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1353/2737] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:53 2022] DELETE /identity/v3/users/0d67a3c60a0a45eebd34e977959bb26d => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:54.520579 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:54.522634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:54.561001 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:54.614993 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b78169-3b2c-4336-8daf-df65acafeec4 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:54.681079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:54.681748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:54.682027 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:54.682313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] PATH_INFO: `/v3/projects/17db2a7a0be94061ab3a0391551316a0` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:54.683576 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17db2a7a0be94061ab3a0391551316a0)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:54.685670 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:54.864166 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17db2a7a0be94061ab3a0391551316a0', 'request_id': 'req-2a667a79-2fff-4da6-aade-d0b3036dd73a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:54.872405 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17db2a7a0be94061ab3a0391551316a0', 'request_id': 'req-2a667a79-2fff-4da6-aade-d0b3036dd73a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.055878 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17db2a7a0be94061ab3a0391551316a0', 'request_id': 'req-2a667a79-2fff-4da6-aade-d0b3036dd73a'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.081124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] The token cache is being invalidate because project 17db2a7a0be94061ab3a0391551316a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:55.081124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.081124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.081124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1385/2738] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:54 2022] DELETE /identity/v3/projects/17db2a7a0be94061ab3a0391551316a0 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:55.081527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:55.081527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:55.097975 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81a17-4a4f-47f6-8e80-52f1373ad88f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:55.099936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:55.102668 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:55.188056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a667a79-2fff-4da6-aade-d0b3036dd73a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:55.197163 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:55.197902 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:55.198249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:55.199199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] PATH_INFO: `/v3/projects/8700209d0c4e44b581b1276f518db6d2` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:55.200480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8700209d0c4e44b581b1276f518db6d2)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:55.202540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:55.202687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:55.203211 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:55.203529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:55.203803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] PATH_INFO: `/v3/users/96c679c2c56046ceb54a8ffd23bc4718` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:55.205021 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c679c2c56046ceb54a8ffd23bc4718)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:55.207048 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:55.394554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8700209d0c4e44b581b1276f518db6d2', 'request_id': 'req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.400303 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8700209d0c4e44b581b1276f518db6d2', 'request_id': 'req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.479819 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c679c2c56046ceb54a8ffd23bc4718'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.480468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c679c2c56046ceb54a8ffd23bc4718'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.492994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c679c2c56046ceb54a8ffd23bc4718'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.493644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c679c2c56046ceb54a8ffd23bc4718'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.500004 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8700209d0c4e44b581b1276f518db6d2', 'request_id': 'req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.512822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] The token cache is being invalidate because project 8700209d0c4e44b581b1276f518db6d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:55.513110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.513923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-3cfcdeb4-0806-4a60-a1ca-1bb65c865c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.515394 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1386/2739] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:55 2022] DELETE /identity/v3/projects/8700209d0c4e44b581b1276f518db6d2 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:55.520209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65ebb2-91b2-436d-9302-8f43a7a811ee None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:55.520573 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65ebb2-91b2-436d-9302-8f43a7a811ee None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:55.522321 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65ebb2-91b2-436d-9302-8f43a7a811ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:55.541205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6e65ebb2-91b2-436d-9302-8f43a7a811ee None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:55.542429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6e65ebb2-91b2-436d-9302-8f43a7a811ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:55.592460 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:55.604654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1387/2740] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:55.613146 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523b4f5-9a98-47c8-80d6-2abadc8a2c60 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:55.613424 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523b4f5-9a98-47c8-80d6-2abadc8a2c60 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:55.613688 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523b4f5-9a98-47c8-80d6-2abadc8a2c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:55.620253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c679c2c56046ceb54a8ffd23bc4718'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:55.635511 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1354/2741] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:55 2022] DELETE /identity/v3/users/96c679c2c56046ceb54a8ffd23bc4718 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:55.636886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-f523b4f5-9a98-47c8-80d6-2abadc8a2c60 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:55.638186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-f523b4f5-9a98-47c8-80d6-2abadc8a2c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:55.704370 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1388/2742] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:55.708696 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:55.710215 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:55.723384 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dee16c-9214-4ee8-ae44-2eb64d1258a8 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:55.728903 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:55.729509 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:55.729788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:55.730070 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:55.730710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:55.731039 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:55.732634 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:55.745166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1355/2743] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:05:55.749746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523b4f5-9a98-47c8-80d6-2abadc8a2c60 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:55.759090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523b4f5-9a98-47c8-80d6-2abadc8a2c60 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:55.763866 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:55.764467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:55.764738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:55.765002 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:55.765632 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:55.767562 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:55.902766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1389/2744] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:55 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:55.906803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:55.917948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e73dc5-5933-49e4-b88e-ef134a047006 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:55.977575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d5c8-5390-4870-a77f-654b2f769442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:55.978229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d5c8-5390-4870-a77f-654b2f769442 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:55.984802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d5c8-5390-4870-a77f-654b2f769442 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:55.984802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d5c8-5390-4870-a77f-654b2f769442 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:55.984802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8d5c8-5390-4870-a77f-654b2f769442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:55.984802 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8d5c8-5390-4870-a77f-654b2f769442 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:56.147601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:56.150575 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1356/2745] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:55 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:56.151116 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:56.151664 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d5c8-5390-4870-a77f-654b2f769442 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:56.168528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a93b4b-8aec-4861-9d7f-9430c7946d51 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:56.168650 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d5c8-5390-4870-a77f-654b2f769442 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:56.170060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:56.170817 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.171135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.171583 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] PATH_INFO: `/v3/projects/0890e892d7814ca88d28264d2308b1a2` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.172606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0890e892d7814ca88d28264d2308b1a2)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:56.174654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:56.174948 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb4972-3265-46b1-a061-b75ce001d369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:56.175652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb4972-3265-46b1-a061-b75ce001d369 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.175927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb4972-3265-46b1-a061-b75ce001d369 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.176392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb4972-3265-46b1-a061-b75ce001d369 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.177042 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb4972-3265-46b1-a061-b75ce001d369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:56.179011 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb4972-3265-46b1-a061-b75ce001d369 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:56.191046 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1357/2746] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:56.194713 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb4972-3265-46b1-a061-b75ce001d369 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:56.205627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb4972-3265-46b1-a061-b75ce001d369 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:56.210285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a2045-3ce7-4d26-8b08-cb1067962fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:56.210872 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a2045-3ce7-4d26-8b08-cb1067962fbc admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.211887 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a2045-3ce7-4d26-8b08-cb1067962fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.212102 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a2045-3ce7-4d26-8b08-cb1067962fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.212911 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a2045-3ce7-4d26-8b08-cb1067962fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:56.214510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a2045-3ce7-4d26-8b08-cb1067962fbc admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:56.224474 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1358/2747] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:56.228858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a2045-3ce7-4d26-8b08-cb1067962fbc admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:56.231554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:56.238644 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a2045-3ce7-4d26-8b08-cb1067962fbc admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:56.243281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca24fa2-1d73-446e-89cf-186990bb1844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:56.243919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca24fa2-1d73-446e-89cf-186990bb1844 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.244192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca24fa2-1d73-446e-89cf-186990bb1844 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.244456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca24fa2-1d73-446e-89cf-186990bb1844 admin admin] PATH_INFO: `/v3/projects/4bb99090463241cc81aa4018dabeacbc/users/14e4b6b47f894cc1af7ca1eb6ec4ea60/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.258987 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca24fa2-1d73-446e-89cf-186990bb1844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb99090463241cc81aa4018dabeacbc, user_id=14e4b6b47f894cc1af7ca1eb6ec4ea60, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:56.261638 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca24fa2-1d73-446e-89cf-186990bb1844 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:56.426731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1359/2748] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:56 2022] PUT /identity/v3/projects/4bb99090463241cc81aa4018dabeacbc/users/14e4b6b47f894cc1af7ca1eb6ec4ea60/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:56.435308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0890e892d7814ca88d28264d2308b1a2', 'request_id': 'req-269e8231-175d-4508-971b-f876461b4116'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:56.438158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51bd1b-be75-40a6-9772-ac0ec202ac23 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.438442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51bd1b-be75-40a6-9772-ac0ec202ac23 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.438710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51bd1b-be75-40a6-9772-ac0ec202ac23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.441317 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0890e892d7814ca88d28264d2308b1a2', 'request_id': 'req-269e8231-175d-4508-971b-f876461b4116'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:56.467123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6f51bd1b-be75-40a6-9772-ac0ec202ac23 None None] MFA Rules not processed for user `14e4b6b47f894cc1af7ca1eb6ec4ea60`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:56.468354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6f51bd1b-be75-40a6-9772-ac0ec202ac23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:56.532790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1360/2749] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:56.534319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0890e892d7814ca88d28264d2308b1a2', 'request_id': 'req-269e8231-175d-4508-971b-f876461b4116'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:56.537331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51bd1b-be75-40a6-9772-ac0ec202ac23 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:56.546687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] The token cache is being invalidate because project 0890e892d7814ca88d28264d2308b1a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:56.547006 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:56.547762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-269e8231-175d-4508-971b-f876461b4116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:56.549372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1390/2750] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:56 2022] DELETE /identity/v3/projects/0890e892d7814ca88d28264d2308b1a2 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:56.560206 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f404f-05bc-4270-b981-8b5e763590aa None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.560937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f404f-05bc-4270-b981-8b5e763590aa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.561247 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f404f-05bc-4270-b981-8b5e763590aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.588368 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-263f404f-05bc-4270-b981-8b5e763590aa None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:56.589588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-263f404f-05bc-4270-b981-8b5e763590aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:56.608271 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51bd1b-be75-40a6-9772-ac0ec202ac23 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:56.612870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba19d8d-9c87-4a71-98e8-a3b946e7ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:56.613472 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba19d8d-9c87-4a71-98e8-a3b946e7ff8f admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.613743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba19d8d-9c87-4a71-98e8-a3b946e7ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.614038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba19d8d-9c87-4a71-98e8-a3b946e7ff8f admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.614652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba19d8d-9c87-4a71-98e8-a3b946e7ff8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:56.616693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba19d8d-9c87-4a71-98e8-a3b946e7ff8f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:56.651863 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1391/2751] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:56.652715 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f404f-05bc-4270-b981-8b5e763590aa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:56.653976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-263f404f-05bc-4270-b981-8b5e763590aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:56.666016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f404f-05bc-4270-b981-8b5e763590aa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:56.673644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:56.673644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.673644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.673644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] PATH_INFO: `/v3/users/5c62754c3bc9492486d44f7b3a3add9b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.674474 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c62754c3bc9492486d44f7b3a3add9b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:56.676534 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:56.741500 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1361/2752] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:56 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:05:56.743284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d8173-6b23-4303-ae85-c9823bb4dcea None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.743499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d8173-6b23-4303-ae85-c9823bb4dcea None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.743767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d8173-6b23-4303-ae85-c9823bb4dcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.763405 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-3c5d8173-6b23-4303-ae85-c9823bb4dcea None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:56.764639 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3c5d8173-6b23-4303-ae85-c9823bb4dcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:56.822249 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1362/2753] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:56.823090 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d8173-6b23-4303-ae85-c9823bb4dcea None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:56.824439 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3c5d8173-6b23-4303-ae85-c9823bb4dcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:56.835944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d8173-6b23-4303-ae85-c9823bb4dcea None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:56.840683 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a6603-eec9-4554-9693-51798d404420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:56.841269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a6603-eec9-4554-9693-51798d404420 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:56.841565 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a6603-eec9-4554-9693-51798d404420 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:56.841834 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a6603-eec9-4554-9693-51798d404420 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:56.842446 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a6603-eec9-4554-9693-51798d404420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:56.844372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a6603-eec9-4554-9693-51798d404420 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:56.913717 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c62754c3bc9492486d44f7b3a3add9b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:56.914757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c62754c3bc9492486d44f7b3a3add9b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:56.928348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c62754c3bc9492486d44f7b3a3add9b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:56.975966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:56.982354 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1363/2754] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:56 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:56.983241 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a6603-eec9-4554-9693-51798d404420 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:56.994966 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a6603-eec9-4554-9693-51798d404420 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:56.999627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.000223 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.000496 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.000762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.001401 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:05:57.001693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.003302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.008603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c62754c3bc9492486d44f7b3a3add9b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:57.015995 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1364/2755] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:05:57.018233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c62754c3bc9492486d44f7b3a3add9b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:57.018410 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.019208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.022596 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1392/2756] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:56 2022] DELETE /identity/v3/users/5c62754c3bc9492486d44f7b3a3add9b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:57.023444 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.024921 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.032479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4220100-44e4-4734-a37d-f46427d59e8b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.090997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b21545-739a-4f9c-beac-7d0b80e50207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.091419 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09435d5b-45af-4062-b4e4-4abcc53fce39 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.091614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b21545-739a-4f9c-beac-7d0b80e50207 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.091892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b21545-739a-4f9c-beac-7d0b80e50207 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.092147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b21545-739a-4f9c-beac-7d0b80e50207 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.093009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b21545-739a-4f9c-beac-7d0b80e50207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.094751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b21545-739a-4f9c-beac-7d0b80e50207 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.096236 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.098626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.098626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.098626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.098626 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.099975 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.108413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1365/2757] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:57.109677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148bb3-ae0a-42ff-8f5d-e4841fa5d66d None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.109961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148bb3-ae0a-42ff-8f5d-e4841fa5d66d None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.110231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148bb3-ae0a-42ff-8f5d-e4841fa5d66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.130416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-ef148bb3-ae0a-42ff-8f5d-e4841fa5d66d None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:57.131659 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ef148bb3-ae0a-42ff-8f5d-e4841fa5d66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.134674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:57.139651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1366/2758] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:57.140411 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef148bb3-ae0a-42ff-8f5d-e4841fa5d66d None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.150203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef148bb3-ae0a-42ff-8f5d-e4841fa5d66d None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.155022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a887db8-2628-43ca-a71d-d0b308d8c4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.155749 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a887db8-2628-43ca-a71d-d0b308d8c4da admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.156079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a887db8-2628-43ca-a71d-d0b308d8c4da admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.156365 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a887db8-2628-43ca-a71d-d0b308d8c4da admin admin] PATH_INFO: `/v3/projects/e548ce8b13724c058e4cfc2d63a73b5d/users/a91575037c6940ceb26953b4268b5080/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.171652 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a887db8-2628-43ca-a71d-d0b308d8c4da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e548ce8b13724c058e4cfc2d63a73b5d, user_id=a91575037c6940ceb26953b4268b5080, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.174448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a887db8-2628-43ca-a71d-d0b308d8c4da admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.210179 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1393/2759] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:05:57 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:57.210851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.212908 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.266385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1367/2760] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:57 2022] PUT /identity/v3/projects/e548ce8b13724c058e4cfc2d63a73b5d/users/a91575037c6940ceb26953b4268b5080/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:57.267187 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a887db8-2628-43ca-a71d-d0b308d8c4da admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.277263 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f57f1-cc03-4682-9bfe-4d3267a9595f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.327363 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:57.327762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614a2df-5424-4960-96f7-6fff9a1e30a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.328413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614a2df-5424-4960-96f7-6fff9a1e30a6 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.328694 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614a2df-5424-4960-96f7-6fff9a1e30a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.328964 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614a2df-5424-4960-96f7-6fff9a1e30a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.332450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a887db8-2628-43ca-a71d-d0b308d8c4da admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.338055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d3f7-8644-4609-970d-97c397294409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.338646 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d3f7-8644-4609-970d-97c397294409 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.338932 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d3f7-8644-4609-970d-97c397294409 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.339216 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d3f7-8644-4609-970d-97c397294409 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.339346 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614a2df-5424-4960-96f7-6fff9a1e30a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.339837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d3f7-8644-4609-970d-97c397294409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.340703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614a2df-5424-4960-96f7-6fff9a1e30a6 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.341739 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d3f7-8644-4609-970d-97c397294409 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.352489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1394/2761] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:57.353619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614a2df-5424-4960-96f7-6fff9a1e30a6 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.363329 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614a2df-5424-4960-96f7-6fff9a1e30a6 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.368254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba00cf-7b1f-4d0a-816a-9618de698a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.369262 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba00cf-7b1f-4d0a-816a-9618de698a9c admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.369533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba00cf-7b1f-4d0a-816a-9618de698a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.369871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba00cf-7b1f-4d0a-816a-9618de698a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.370506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba00cf-7b1f-4d0a-816a-9618de698a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.372109 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba00cf-7b1f-4d0a-816a-9618de698a9c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.382497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1395/2762] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:05:57.386766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba00cf-7b1f-4d0a-816a-9618de698a9c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.396519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba00cf-7b1f-4d0a-816a-9618de698a9c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.402300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29849f-0ef6-4c15-ac0c-4d381d0a6514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.403190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29849f-0ef6-4c15-ac0c-4d381d0a6514 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.403463 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29849f-0ef6-4c15-ac0c-4d381d0a6514 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.403729 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29849f-0ef6-4c15-ac0c-4d381d0a6514 admin admin] PATH_INFO: `/v3/projects/e548ce8b13724c058e4cfc2d63a73b5d/users/a91575037c6940ceb26953b4268b5080/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.405918 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29849f-0ef6-4c15-ac0c-4d381d0a6514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e548ce8b13724c058e4cfc2d63a73b5d, user_id=a91575037c6940ceb26953b4268b5080, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.408784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29849f-0ef6-4c15-ac0c-4d381d0a6514 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.420650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1396/2763] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:57 2022] PUT /identity/v3/projects/e548ce8b13724c058e4cfc2d63a73b5d/users/a91575037c6940ceb26953b4268b5080/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:57.431704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d1ec-0c05-4924-94d0-235e7ac4edab None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.431992 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d1ec-0c05-4924-94d0-235e7ac4edab None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.432264 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472d1ec-0c05-4924-94d0-235e7ac4edab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.443808 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1368/2764] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:05:57 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:05:57.448801 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d3f7-8644-4609-970d-97c397294409 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.460115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d3f7-8644-4609-970d-97c397294409 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.463216 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1472d1ec-0c05-4924-94d0-235e7ac4edab None None] MFA Rules not processed for user `a91575037c6940ceb26953b4268b5080`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:57.464479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1472d1ec-0c05-4924-94d0-235e7ac4edab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.491016 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:57.517370 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c34516-bb15-4af3-9061-44dd3d304bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.517910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34516-bb15-4af3-9061-44dd3d304bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.518183 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34516-bb15-4af3-9061-44dd3d304bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.518651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34516-bb15-4af3-9061-44dd3d304bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.519378 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c34516-bb15-4af3-9061-44dd3d304bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.520993 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c34516-bb15-4af3-9061-44dd3d304bd9 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.533350 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1369/2765] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:57.537343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c34516-bb15-4af3-9061-44dd3d304bd9 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.538175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1397/2766] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:57.550426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c34516-bb15-4af3-9061-44dd3d304bd9 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.551995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472d1ec-0c05-4924-94d0-235e7ac4edab None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.553415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1472d1ec-0c05-4924-94d0-235e7ac4edab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.556150 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc63a2-0096-4aa9-9a43-adf2757c490f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.557382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc63a2-0096-4aa9-9a43-adf2757c490f admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.557382 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc63a2-0096-4aa9-9a43-adf2757c490f admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.557611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc63a2-0096-4aa9-9a43-adf2757c490f admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.558434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc63a2-0096-4aa9-9a43-adf2757c490f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.566088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc63a2-0096-4aa9-9a43-adf2757c490f admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.578780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1370/2767] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:05:57.581349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc63a2-0096-4aa9-9a43-adf2757c490f admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.594445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc63a2-0096-4aa9-9a43-adf2757c490f admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.600868 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.601733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.603044 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.603348 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] PATH_INFO: `/v3/projects/43908612ede248f48a57ec5f38f207e7/users/8c136ff804ba4430b86b65867f5c197b/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.607111 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:57.620107 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43908612ede248f48a57ec5f38f207e7, user_id=8c136ff804ba4430b86b65867f5c197b, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.622759 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.628417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472d1ec-0c05-4924-94d0-235e7ac4edab None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.628417 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905132d0-af0a-42c3-beeb-4f1371cbd2c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.628764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905132d0-af0a-42c3-beeb-4f1371cbd2c3 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.629047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905132d0-af0a-42c3-beeb-4f1371cbd2c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.629342 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905132d0-af0a-42c3-beeb-4f1371cbd2c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.631599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-905132d0-af0a-42c3-beeb-4f1371cbd2c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.694398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905132d0-af0a-42c3-beeb-4f1371cbd2c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.696449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905132d0-af0a-42c3-beeb-4f1371cbd2c3 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.705483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1398/2768] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:05:57.757680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1371/2769] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:05:57 2022] PUT /identity/v3/projects/43908612ede248f48a57ec5f38f207e7/users/8c136ff804ba4430b86b65867f5c197b/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:57.769603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9274-0899-46df-9048-d575e852d8c7 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.770265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9274-0899-46df-9048-d575e852d8c7 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.770265 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d9274-0899-46df-9048-d575e852d8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.801157 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ff6d9274-0899-46df-9048-d575e852d8c7 None None] MFA Rules not processed for user `8c136ff804ba4430b86b65867f5c197b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:57.802434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ff6d9274-0899-46df-9048-d575e852d8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.851322 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.853611 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:57.886034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1399/2770] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:57.897784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d9274-0899-46df-9048-d575e852d8c7 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:57.939001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71297f6-2c26-41a3-b5d0-53c963beb210 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.945118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.946668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.947191 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.947668 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] PATH_INFO: `/v3/projects/ae1d1f686c5147518ba5b391af2f1a83` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.949685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae1d1f686c5147518ba5b391af2f1a83)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.952297 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.958386 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d9274-0899-46df-9048-d575e852d8c7 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:57.964762 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bdce9-5505-4a25-a9a8-6bc48512509f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:57.965634 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bdce9-5505-4a25-a9a8-6bc48512509f service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:57.965887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bdce9-5505-4a25-a9a8-6bc48512509f service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:57.968257 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bdce9-5505-4a25-a9a8-6bc48512509f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:57.976705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bdce9-5505-4a25-a9a8-6bc48512509f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:57.978151 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bdce9-5505-4a25-a9a8-6bc48512509f service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:57.987660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1400/2771] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:58.007589 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bdce9-5505-4a25-a9a8-6bc48512509f service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:58.019259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bdce9-5505-4a25-a9a8-6bc48512509f service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:58.025145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:58.025701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:58.025976 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:58.026244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] PATH_INFO: `/v3/users/8c136ff804ba4430b86b65867f5c197b` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:58.027441 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c136ff804ba4430b86b65867f5c197b)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:58.029348 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:58.078529 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1d1f686c5147518ba5b391af2f1a83', 'request_id': 'req-a134b059-700d-4924-b129-78a5c68c1168'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.083908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1d1f686c5147518ba5b391af2f1a83', 'request_id': 'req-a134b059-700d-4924-b129-78a5c68c1168'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.213450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1d1f686c5147518ba5b391af2f1a83', 'request_id': 'req-a134b059-700d-4924-b129-78a5c68c1168'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.225600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] The token cache is being invalidate because project ae1d1f686c5147518ba5b391af2f1a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:58.225879 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.226590 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.228159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1372/2772] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:57 2022] DELETE /identity/v3/projects/ae1d1f686c5147518ba5b391af2f1a83 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:58.232053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:58.233387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:58.283137 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:58.311018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134b059-700d-4924-b129-78a5c68c1168 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:58.316154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:58.316780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:58.317052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:58.317316 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] PATH_INFO: `/v3/users/f267bf925f7b40d1be7d5f5c14219096` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:58.318428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f267bf925f7b40d1be7d5f5c14219096)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:58.320990 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:58.336457 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:58.520351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c136ff804ba4430b86b65867f5c197b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.527423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c136ff804ba4430b86b65867f5c197b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.535416 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:58.544041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c136ff804ba4430b86b65867f5c197b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.551416 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f267bf925f7b40d1be7d5f5c14219096'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.552069 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f267bf925f7b40d1be7d5f5c14219096'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.565572 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f267bf925f7b40d1be7d5f5c14219096'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.566273 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f267bf925f7b40d1be7d5f5c14219096'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.654700 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c136ff804ba4430b86b65867f5c197b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.662650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-f696f299-fcf5-4d66-bae2-6ec1aa0c4b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c136ff804ba4430b86b65867f5c197b'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.669332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1401/2773] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:58 2022] DELETE /identity/v3/users/8c136ff804ba4430b86b65867f5c197b => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:58.673557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3868c-89fb-4408-93d3-2bac73bad6ac None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:58.673843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3868c-89fb-4408-93d3-2bac73bad6ac None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:58.674156 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3868c-89fb-4408-93d3-2bac73bad6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:58.693542 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-14e3868c-89fb-4408-93d3-2bac73bad6ac None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:58.694738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-14e3868c-89fb-4408-93d3-2bac73bad6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:58.753722 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1402/2774] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:05:58.763561 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3868c-89fb-4408-93d3-2bac73bad6ac None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:58.764888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-14e3868c-89fb-4408-93d3-2bac73bad6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:58.768940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f267bf925f7b40d1be7d5f5c14219096'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:58.770167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:58.783720 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1373/2775] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:05:58 2022] DELETE /identity/v3/users/f267bf925f7b40d1be7d5f5c14219096 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:05:58.827210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3868c-89fb-4408-93d3-2bac73bad6ac None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:58.880135 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:58.880785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:58.881065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:58.881339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:58.942190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:58.943675 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:58.953336 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1403/2776] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:05:59.322205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:59.322205 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:59.329576 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57e7ca-e10c-4197-8ea5-109d0dcb5320 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:59.337124 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:59.337781 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:59.338053 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:59.338320 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] PATH_INFO: `/v3/projects/4fdb2e3c07aa475a952ecac0d31e4994` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:59.339536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fdb2e3c07aa475a952ecac0d31e4994)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:59.341433 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:59.473108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:59.474669 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:59.488242 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fdb2e3c07aa475a952ecac0d31e4994', 'request_id': 'req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.496268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618acad-33df-49a5-95c0-067faeb0131e service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:05:59.499886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fdb2e3c07aa475a952ecac0d31e4994', 'request_id': 'req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.557112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:05:59.557471 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:59.557826 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:59.558022 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] PATH_INFO: `/v3/projects/43908612ede248f48a57ec5f38f207e7` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:59.559173 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43908612ede248f48a57ec5f38f207e7)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:05:59.561177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:05:59.572833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=100 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:05:59.574688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fdb2e3c07aa475a952ecac0d31e4994', 'request_id': 'req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.587767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] The token cache is being invalidate because project 4fdb2e3c07aa475a952ecac0d31e4994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:59.588050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.588714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-6c6dc6c5-79b8-4659-ad64-dd3f9db2c362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.590532 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1374/2777] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:59 2022] DELETE /identity/v3/projects/4fdb2e3c07aa475a952ecac0d31e4994 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:59.712955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43908612ede248f48a57ec5f38f207e7', 'request_id': 'req-542e662c-204f-4d16-a5be-ec440ef172a4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.720880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43908612ede248f48a57ec5f38f207e7', 'request_id': 'req-542e662c-204f-4d16-a5be-ec440ef172a4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.841479 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43908612ede248f48a57ec5f38f207e7', 'request_id': 'req-542e662c-204f-4d16-a5be-ec440ef172a4'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.846539 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] The token cache is being invalidate because project 43908612ede248f48a57ec5f38f207e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:05:59.846832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.847569 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-542e662c-204f-4d16-a5be-ec440ef172a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:05:59.849550 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1404/2778] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:05:59 2022] DELETE /identity/v3/projects/43908612ede248f48a57ec5f38f207e7 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:05:59.855482 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aa974-7833-46a6-ac3a-2102ae96c921 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:59.855768 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aa974-7833-46a6-ac3a-2102ae96c921 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:59.856038 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aa974-7833-46a6-ac3a-2102ae96c921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:59.875694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-830aa974-7833-46a6-ac3a-2102ae96c921 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:59.876886 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-830aa974-7833-46a6-ac3a-2102ae96c921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:59.941101 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1375/2779] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:59.950028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a2fa-8c36-494e-bf77-2ed13787a597 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:05:59.950315 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a2fa-8c36-494e-bf77-2ed13787a597 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:05:59.950615 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663a2fa-8c36-494e-bf77-2ed13787a597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:05:59.971849 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-d663a2fa-8c36-494e-bf77-2ed13787a597 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:05:59.979155 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d663a2fa-8c36-494e-bf77-2ed13787a597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:05:59.981619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1405/2780] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:05:59.985917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830aa974-7833-46a6-ac3a-2102ae96c921 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:05:59.996007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830aa974-7833-46a6-ac3a-2102ae96c921 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:00.002077 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:00.008374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:00.008374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:00.008374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:00.008374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:06:00.008374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:00.008374 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:00.018578 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1376/2781] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:06:00.022847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663a2fa-8c36-494e-bf77-2ed13787a597 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:00.033650 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663a2fa-8c36-494e-bf77-2ed13787a597 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:00.037686 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332950fe-bb54-46e8-8c89-b0e6e072796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:00.038280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332950fe-bb54-46e8-8c89-b0e6e072796c admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:00.038551 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332950fe-bb54-46e8-8c89-b0e6e072796c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:00.038812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332950fe-bb54-46e8-8c89-b0e6e072796c admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:00.039564 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332950fe-bb54-46e8-8c89-b0e6e072796c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:00.041475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332950fe-bb54-46e8-8c89-b0e6e072796c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:00.188047 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1406/2782] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:06:00 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:00.192602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:00.200231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:00.204280 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5841aa8-3289-44ed-aa46-8d4483653b14 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:00.262442 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a5be-2de5-4a25-8d13-d26f0c171e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:00.263080 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a5be-2de5-4a25-8d13-d26f0c171e66 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:00.263358 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a5be-2de5-4a25-8d13-d26f0c171e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:00.263625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a5be-2de5-4a25-8d13-d26f0c171e66 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:00.264282 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0a5be-2de5-4a25-8d13-d26f0c171e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:00.266226 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0a5be-2de5-4a25-8d13-d26f0c171e66 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:00.397417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1377/2783] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:06:00 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:00.401788 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332950fe-bb54-46e8-8c89-b0e6e072796c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:00.412210 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332950fe-bb54-46e8-8c89-b0e6e072796c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:00.416709 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc1c2f-e103-4663-b08d-5fb0f6ff013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:00.417300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc1c2f-e103-4663-b08d-5fb0f6ff013f admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:00.417601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc1c2f-e103-4663-b08d-5fb0f6ff013f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:00.417927 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc1c2f-e103-4663-b08d-5fb0f6ff013f admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:00.418663 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc1c2f-e103-4663-b08d-5fb0f6ff013f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:00.420241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc1c2f-e103-4663-b08d-5fb0f6ff013f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:00.434685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1407/2784] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:00.436341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a5be-2de5-4a25-8d13-d26f0c171e66 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:00.457285 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a5be-2de5-4a25-8d13-d26f0c171e66 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:00.462154 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931f3ba-31a4-463a-9cd2-a0765201373e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:00.463437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931f3ba-31a4-463a-9cd2-a0765201373e admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:00.463717 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931f3ba-31a4-463a-9cd2-a0765201373e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:00.464172 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931f3ba-31a4-463a-9cd2-a0765201373e admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:00.464714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931f3ba-31a4-463a-9cd2-a0765201373e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:00.466773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931f3ba-31a4-463a-9cd2-a0765201373e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:00.478545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1378/2785] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:00.482769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc1c2f-e103-4663-b08d-5fb0f6ff013f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:00.493987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc1c2f-e103-4663-b08d-5fb0f6ff013f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:00.498540 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bca25-6862-4558-a7b0-7b8814405ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:00.499217 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bca25-6862-4558-a7b0-7b8814405ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:00.499494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bca25-6862-4558-a7b0-7b8814405ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:00.499764 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bca25-6862-4558-a7b0-7b8814405ae1 admin admin] PATH_INFO: `/v3/projects/abfb63c67c3b48b8b4c0bdec2f68d2b4/users/2a819f15280e4a32b28534768e5d6e57/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:00.515512 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bca25-6862-4558-a7b0-7b8814405ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abfb63c67c3b48b8b4c0bdec2f68d2b4, user_id=2a819f15280e4a32b28534768e5d6e57, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:00.518158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bca25-6862-4558-a7b0-7b8814405ae1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:00.631800 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1408/2786] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:06:00 2022] PUT /identity/v3/projects/abfb63c67c3b48b8b4c0bdec2f68d2b4/users/2a819f15280e4a32b28534768e5d6e57/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:00.643943 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9b63e-482b-4316-b01b-0db597f1d585 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:00.644229 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9b63e-482b-4316-b01b-0db597f1d585 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:00.644504 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9b63e-482b-4316-b01b-0db597f1d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:00.674071 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-dfc9b63e-482b-4316-b01b-0db597f1d585 None None] MFA Rules not processed for user `2a819f15280e4a32b28534768e5d6e57`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:00.675409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-dfc9b63e-482b-4316-b01b-0db597f1d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:00.741571 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1379/2787] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:00.746944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bca25-6862-4558-a7b0-7b8814405ae1 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:00.809339 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bca25-6862-4558-a7b0-7b8814405ae1 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:00.814453 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81210-4ed0-4259-927d-7337b19705e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:00.815078 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81210-4ed0-4259-927d-7337b19705e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:00.815354 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81210-4ed0-4259-927d-7337b19705e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:00.815619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81210-4ed0-4259-927d-7337b19705e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:00.816228 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81210-4ed0-4259-927d-7337b19705e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:00.818413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81210-4ed0-4259-927d-7337b19705e1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:00.930319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1409/2788] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:06:00 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:00.934616 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9b63e-482b-4316-b01b-0db597f1d585 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:00.946275 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9b63e-482b-4316-b01b-0db597f1d585 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:01.005239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3698503-1530-43e0-979d-29ea1aeb77e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:01.005837 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698503-1530-43e0-979d-29ea1aeb77e5 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.006139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698503-1530-43e0-979d-29ea1aeb77e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.006376 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698503-1530-43e0-979d-29ea1aeb77e5 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.007062 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3698503-1530-43e0-979d-29ea1aeb77e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:01.009055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3698503-1530-43e0-979d-29ea1aeb77e5 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:01.116627 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1380/2789] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:06:00 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:01.120782 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81210-4ed0-4259-927d-7337b19705e1 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:01.131550 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81210-4ed0-4259-927d-7337b19705e1 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:01.136259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6744bf-ef77-4077-bf24-19f12f68d932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:01.136846 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6744bf-ef77-4077-bf24-19f12f68d932 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.137140 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6744bf-ef77-4077-bf24-19f12f68d932 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.137403 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6744bf-ef77-4077-bf24-19f12f68d932 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.138028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6744bf-ef77-4077-bf24-19f12f68d932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:01.139832 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6744bf-ef77-4077-bf24-19f12f68d932 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:01.151254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1410/2790] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:01.155252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3698503-1530-43e0-979d-29ea1aeb77e5 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:01.161914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:01.167203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3698503-1530-43e0-979d-29ea1aeb77e5 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:01.171927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11746f0-ce1d-44b2-818f-16cb6873d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:01.172520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11746f0-ce1d-44b2-818f-16cb6873d49d admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.172799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11746f0-ce1d-44b2-818f-16cb6873d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.173076 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11746f0-ce1d-44b2-818f-16cb6873d49d admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.173714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11746f0-ce1d-44b2-818f-16cb6873d49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:01.175341 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11746f0-ce1d-44b2-818f-16cb6873d49d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:01.185458 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1381/2791] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:01.190131 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6744bf-ef77-4077-bf24-19f12f68d932 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:01.200246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6744bf-ef77-4077-bf24-19f12f68d932 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:01.204845 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7754286-9000-4642-80c1-95e3830a573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:01.205488 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754286-9000-4642-80c1-95e3830a573d admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.205766 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754286-9000-4642-80c1-95e3830a573d admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.206033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7754286-9000-4642-80c1-95e3830a573d admin admin] PATH_INFO: `/v3/projects/c7165d3f6d014b528e97a07a2056c0b0/users/a6999b66901e49638e5979843209bc98/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.209122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:01.221737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7754286-9000-4642-80c1-95e3830a573d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7165d3f6d014b528e97a07a2056c0b0, user_id=a6999b66901e49638e5979843209bc98, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:01.228090 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7754286-9000-4642-80c1-95e3830a573d admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:01.323067 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1411/2792] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:06:01 2022] PUT /identity/v3/projects/c7165d3f6d014b528e97a07a2056c0b0/users/a6999b66901e49638e5979843209bc98/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:06:01.333724 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de42d08-c834-46a6-bba1-bfe52e3e3378 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.334009 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de42d08-c834-46a6-bba1-bfe52e3e3378 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.334283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de42d08-c834-46a6-bba1-bfe52e3e3378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.363645 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-6de42d08-c834-46a6-bba1-bfe52e3e3378 None None] MFA Rules not processed for user `a6999b66901e49638e5979843209bc98`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:01.364864 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-6de42d08-c834-46a6-bba1-bfe52e3e3378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:01.430621 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1382/2793] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:01.435187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7754286-9000-4642-80c1-95e3830a573d admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:01.498635 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7754286-9000-4642-80c1-95e3830a573d admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:01.503378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe0575-450d-47d9-8dff-7bc5c46591f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:01.504028 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe0575-450d-47d9-8dff-7bc5c46591f2 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.504300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe0575-450d-47d9-8dff-7bc5c46591f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.505012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe0575-450d-47d9-8dff-7bc5c46591f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.505175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe0575-450d-47d9-8dff-7bc5c46591f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:01.507111 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe0575-450d-47d9-8dff-7bc5c46591f2 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:01.597234 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1412/2794] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:06:01 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:01.598467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de42d08-c834-46a6-bba1-bfe52e3e3378 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:01.609803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de42d08-c834-46a6-bba1-bfe52e3e3378 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:01.668666 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe026-1eb0-4d37-bfcc-9cef3e13cc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:01.669294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe026-1eb0-4d37-bfcc-9cef3e13cc93 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.669558 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe026-1eb0-4d37-bfcc-9cef3e13cc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.669829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffe026-1eb0-4d37-bfcc-9cef3e13cc93 admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.670447 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffe026-1eb0-4d37-bfcc-9cef3e13cc93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:01.672397 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffe026-1eb0-4d37-bfcc-9cef3e13cc93 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:01.786018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1383/2795] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:06:01 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:01.790142 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe0575-450d-47d9-8dff-7bc5c46591f2 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:01.800460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe0575-450d-47d9-8dff-7bc5c46591f2 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:01.805757 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841f950-e07f-4c40-83c3-03570b44d68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:01.806343 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841f950-e07f-4c40-83c3-03570b44d68b admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.806619 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841f950-e07f-4c40-83c3-03570b44d68b admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.806970 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841f950-e07f-4c40-83c3-03570b44d68b admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.807610 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841f950-e07f-4c40-83c3-03570b44d68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:01.810345 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841f950-e07f-4c40-83c3-03570b44d68b admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:01.822774 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1413/2796] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:01.828673 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe026-1eb0-4d37-bfcc-9cef3e13cc93 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:01.839541 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffe026-1eb0-4d37-bfcc-9cef3e13cc93 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:01.844306 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a8e33-3df2-4a98-8099-1f51e548d590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:01.844958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a8e33-3df2-4a98-8099-1f51e548d590 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:01.845234 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a8e33-3df2-4a98-8099-1f51e548d590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:01.845503 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a8e33-3df2-4a98-8099-1f51e548d590 admin admin] PATH_INFO: `/v3/projects/9ffa8acac21b46b48f92da6823f16e89/users/97221877843546009e15c96ab204e4fd/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:01.860415 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a8e33-3df2-4a98-8099-1f51e548d590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffa8acac21b46b48f92da6823f16e89, user_id=97221877843546009e15c96ab204e4fd, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:01.865762 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a8e33-3df2-4a98-8099-1f51e548d590 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:01.975445 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1384/2797] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:06:01 2022] PUT /identity/v3/projects/9ffa8acac21b46b48f92da6823f16e89/users/97221877843546009e15c96ab204e4fd/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:01.982400 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841f950-e07f-4c40-83c3-03570b44d68b admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:01.989806 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841f950-e07f-4c40-83c3-03570b44d68b admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:02.049747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c12a8c-bd48-4475-a296-935b67bd38a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:02.050291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c12a8c-bd48-4475-a296-935b67bd38a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:02.050571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c12a8c-bd48-4475-a296-935b67bd38a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:02.050843 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c12a8c-bd48-4475-a296-935b67bd38a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:02.051556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c12a8c-bd48-4475-a296-935b67bd38a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:02.053180 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c12a8c-bd48-4475-a296-935b67bd38a6 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:02.063777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1414/2798] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:06:02.068312 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a8e33-3df2-4a98-8099-1f51e548d590 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:02.079617 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a8e33-3df2-4a98-8099-1f51e548d590 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:02.084869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:02.092029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:02.092029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:02.092029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] PATH_INFO: `/v3/projects/9ffa8acac21b46b48f92da6823f16e89/users/97221877843546009e15c96ab204e4fd/roles/46d63829902d4da196c41ff99b2ac0c5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:02.092029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffa8acac21b46b48f92da6823f16e89, user_id=97221877843546009e15c96ab204e4fd, role_id=46d63829902d4da196c41ff99b2ac0c5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:02.092029 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:02.101169 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1385/2799] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:06:02 2022] PUT /identity/v3/projects/9ffa8acac21b46b48f92da6823f16e89/users/97221877843546009e15c96ab204e4fd/roles/46d63829902d4da196c41ff99b2ac0c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:02.111728 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce75d-2877-478f-97b6-715d666af5fa None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:02.112033 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce75d-2877-478f-97b6-715d666af5fa None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:02.112320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce75d-2877-478f-97b6-715d666af5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:02.141809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-687ce75d-2877-478f-97b6-715d666af5fa None None] MFA Rules not processed for user `97221877843546009e15c96ab204e4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:02.143075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-687ce75d-2877-478f-97b6-715d666af5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:02.188335 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:02.214021 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1415/2800] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:02.225494 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:02.227581 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:02.272118 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:02.295876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2dcf2-b903-4774-8bff-c48427835880 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:02.299876 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5b1a-3f39-400b-aef0-958be73dd541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:02.300419 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5b1a-3f39-400b-aef0-958be73dd541 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:02.300686 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5b1a-3f39-400b-aef0-958be73dd541 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:02.301051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5b1a-3f39-400b-aef0-958be73dd541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:02.359728 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca5b1a-3f39-400b-aef0-958be73dd541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:02.361158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca5b1a-3f39-400b-aef0-958be73dd541 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:02.370318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1386/2801] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:03.417454 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ce75d-2877-478f-97b6-715d666af5fa None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:03.427372 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ce75d-2877-478f-97b6-715d666af5fa None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:03.432013 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:03.432644 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:03.432913 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:03.433187 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:03.435205 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:03.500267 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:03.501714 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:03.512048 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1416/2802] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:04.517398 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5b1a-3f39-400b-aef0-958be73dd541 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:04.526788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5b1a-3f39-400b-aef0-958be73dd541 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:04.530557 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:04.531252 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:04.535747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:04.535747 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:04.594942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:04.596385 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:04.605381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1387/2803] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:06.182379 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:06.183887 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:06.195708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffd7da-c772-4c18-9329-38c065d88945 service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:06.199397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382a828-6182-40c9-8ff4-5b9d3c4cdc67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:06.200059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a828-6182-40c9-8ff4-5b9d3c4cdc67 service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:06.200337 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a828-6182-40c9-8ff4-5b9d3c4cdc67 service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:06.200607 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a828-6182-40c9-8ff4-5b9d3c4cdc67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:06.209847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382a828-6182-40c9-8ff4-5b9d3c4cdc67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:06.212353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382a828-6182-40c9-8ff4-5b9d3c4cdc67 service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:06.221994 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1417/2804] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:06:09.731763 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:09.733269 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:09.807181 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd2f43-688c-4188-996c-6f63db43d365 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:09.812584 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:09.813209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:09.813479 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:09.813751 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] PATH_INFO: `/v3/users/14e4b6b47f894cc1af7ca1eb6ec4ea60` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:09.820880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e4b6b47f894cc1af7ca1eb6ec4ea60)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:09.820880 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:09.930342 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:10.051091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e4b6b47f894cc1af7ca1eb6ec4ea60'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:10.051750 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e4b6b47f894cc1af7ca1eb6ec4ea60'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:10.064404 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e4b6b47f894cc1af7ca1eb6ec4ea60'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:10.064964 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e4b6b47f894cc1af7ca1eb6ec4ea60'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:10.144803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e4b6b47f894cc1af7ca1eb6ec4ea60'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:10.157400 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1388/2805] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:09 2022] DELETE /identity/v3/users/14e4b6b47f894cc1af7ca1eb6ec4ea60 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:10.161971 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca3065-0241-46b9-b6b0-07fc91deda65 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:10.162470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca3065-0241-46b9-b6b0-07fc91deda65 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:10.162571 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca3065-0241-46b9-b6b0-07fc91deda65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:10.182064 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-0bca3065-0241-46b9-b6b0-07fc91deda65 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:10.184034 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0bca3065-0241-46b9-b6b0-07fc91deda65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:10.247483 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1418/2806] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:10.257305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:10.258702 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:10.318413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00517782-55aa-4bdd-953c-f49d7049a9e6 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:10.322068 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:10.322667 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:10.323018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:10.323527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:10.332767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:10.334194 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:10.344110 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1389/2807] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:10.781255 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca3065-0241-46b9-b6b0-07fc91deda65 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:10.785525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0bca3065-0241-46b9-b6b0-07fc91deda65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:10.795145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca3065-0241-46b9-b6b0-07fc91deda65 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:10.799740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:10.800392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:10.801250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:10.801250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] PATH_INFO: `/v3/projects/4bb99090463241cc81aa4018dabeacbc` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:10.802107 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb99090463241cc81aa4018dabeacbc)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:10.804170 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:10.928250 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb99090463241cc81aa4018dabeacbc', 'request_id': 'req-fada588e-3ae9-45ec-80a5-bf23e9599a21'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:10.933066 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:10.934242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb99090463241cc81aa4018dabeacbc', 'request_id': 'req-fada588e-3ae9-45ec-80a5-bf23e9599a21'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.028849 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:11.030278 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:11.042942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f58d3-f2b3-4fe6-99fe-ee7108b132f1 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:11.049678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb99090463241cc81aa4018dabeacbc', 'request_id': 'req-fada588e-3ae9-45ec-80a5-bf23e9599a21'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.063141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] The token cache is being invalidate because project 4bb99090463241cc81aa4018dabeacbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:11.063510 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.064582 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.066244 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1419/2808] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:10 2022] DELETE /identity/v3/projects/4bb99090463241cc81aa4018dabeacbc => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:11.070322 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:11.071846 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:11.106387 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:11.107060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:11.107333 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:11.107602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] PATH_INFO: `/v3/users/2a819f15280e4a32b28534768e5d6e57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:11.108796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a819f15280e4a32b28534768e5d6e57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:11.110741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:11.140907 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada588e-3ae9-45ec-80a5-bf23e9599a21 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:11.145525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:11.146148 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:11.146420 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:11.146685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] PATH_INFO: `/v3/users/a91575037c6940ceb26953b4268b5080` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:11.147836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91575037c6940ceb26953b4268b5080)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:11.149743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:11.325522 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a819f15280e4a32b28534768e5d6e57'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.326155 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a819f15280e4a32b28534768e5d6e57'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.328219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:11.338803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a819f15280e4a32b28534768e5d6e57'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.341413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a819f15280e4a32b28534768e5d6e57'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.353108 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91575037c6940ceb26953b4268b5080'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.353785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91575037c6940ceb26953b4268b5080'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.366192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91575037c6940ceb26953b4268b5080'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.396495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-65731846-89ec-4340-9f92-ded525149f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a819f15280e4a32b28534768e5d6e57'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.410209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1390/2809] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:11 2022] DELETE /identity/v3/users/2a819f15280e4a32b28534768e5d6e57 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:06:11.414677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fba8b-8de1-4e7f-a150-0b7b772de678 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:11.415013 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fba8b-8de1-4e7f-a150-0b7b772de678 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:11.415315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fba8b-8de1-4e7f-a150-0b7b772de678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:11.441131 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-a61fba8b-8de1-4e7f-a150-0b7b772de678 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:11.442343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a61fba8b-8de1-4e7f-a150-0b7b772de678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:11.453438 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91575037c6940ceb26953b4268b5080'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.461054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91575037c6940ceb26953b4268b5080'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:11.467406 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1420/2810] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:11 2022] DELETE /identity/v3/users/a91575037c6940ceb26953b4268b5080 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:11.504377 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1391/2811] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:06:11.515124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:11.516725 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:11.585744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a9c5b-9cd0-451a-a16f-ac6ad85dc23c admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:11.589677 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:11.590282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:11.590565 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:11.590831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:11.609014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:11.653391 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:11.654831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:11.664948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1421/2812] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:06:11.873251 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fba8b-8de1-4e7f-a150-0b7b772de678 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:11.875538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a61fba8b-8de1-4e7f-a150-0b7b772de678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:11.893645 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fba8b-8de1-4e7f-a150-0b7b772de678 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:11.898570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:11.899462 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:11.899733 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:11.900001 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] PATH_INFO: `/v3/projects/e548ce8b13724c058e4cfc2d63a73b5d` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:11.901923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e548ce8b13724c058e4cfc2d63a73b5d)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:11.904190 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.042495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e548ce8b13724c058e4cfc2d63a73b5d', 'request_id': 'req-83fb69d0-aefc-4304-a27a-d23381a10c36'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.048202 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e548ce8b13724c058e4cfc2d63a73b5d', 'request_id': 'req-83fb69d0-aefc-4304-a27a-d23381a10c36'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.130699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:12.132227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:12.145691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bd7fb-9f07-42fc-86e4-cce8f231af98 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:12.171780 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e548ce8b13724c058e4cfc2d63a73b5d', 'request_id': 'req-83fb69d0-aefc-4304-a27a-d23381a10c36'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.183699 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] The token cache is being invalidate because project e548ce8b13724c058e4cfc2d63a73b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:12.183989 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.184641 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-83fb69d0-aefc-4304-a27a-d23381a10c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.186120 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1392/2813] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:11 2022] DELETE /identity/v3/projects/e548ce8b13724c058e4cfc2d63a73b5d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:12.192083 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eca7d-b400-43c9-9748-73107dfb83b1 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.192361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eca7d-b400-43c9-9748-73107dfb83b1 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.192629 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eca7d-b400-43c9-9748-73107dfb83b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.211757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-946eca7d-b400-43c9-9748-73107dfb83b1 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:12.212961 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-946eca7d-b400-43c9-9748-73107dfb83b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:12.214435 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:12.215580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.215851 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.216122 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] PATH_INFO: `/v3/projects/abfb63c67c3b48b8b4c0bdec2f68d2b4` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.217259 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] RBAC: Authorizing `identity:delete_project(project_id=abfb63c67c3b48b8b4c0bdec2f68d2b4)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:12.219188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.221723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1393/2814] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:06:12.229931 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6be7-0f4e-42e1-83a6-231e4df5c791 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.230207 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6be7-0f4e-42e1-83a6-231e4df5c791 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.230498 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da6be7-0f4e-42e1-83a6-231e4df5c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.251318 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f3da6be7-0f4e-42e1-83a6-231e4df5c791 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:12.252589 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f3da6be7-0f4e-42e1-83a6-231e4df5c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:12.260338 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1394/2815] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:06:12.265526 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da6be7-0f4e-42e1-83a6-231e4df5c791 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:12.275130 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da6be7-0f4e-42e1-83a6-231e4df5c791 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:12.280502 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:12.281104 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.281380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.281648 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.282310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:06:12.282604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:12.284231 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.295788 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1395/2816] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 16:06:12.299968 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:12.309489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41ea1d-f64c-4f66-b488-4d57e5aba797 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:12.314331 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbe17d-8648-4b36-85eb-2032b1e3d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:12.314951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe17d-8648-4b36-85eb-2032b1e3d259 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.315239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe17d-8648-4b36-85eb-2032b1e3d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.315507 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe17d-8648-4b36-85eb-2032b1e3d259 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.316100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbe17d-8648-4b36-85eb-2032b1e3d259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:12.317951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbe17d-8648-4b36-85eb-2032b1e3d259 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.352214 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abfb63c67c3b48b8b4c0bdec2f68d2b4', 'request_id': 'req-55130d2b-da4a-47e3-bd5c-20655bbb577f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.358080 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abfb63c67c3b48b8b4c0bdec2f68d2b4', 'request_id': 'req-55130d2b-da4a-47e3-bd5c-20655bbb577f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.432777 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1396/2817] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:06:12 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:06:12.436505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abfb63c67c3b48b8b4c0bdec2f68d2b4', 'request_id': 'req-55130d2b-da4a-47e3-bd5c-20655bbb577f'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.436892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbe17d-8648-4b36-85eb-2032b1e3d259 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:12.442079 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:12.454983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] The token cache is being invalidate because project abfb63c67c3b48b8b4c0bdec2f68d2b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:12.455275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.455931 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.457491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1422/2818] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:12 2022] DELETE /identity/v3/projects/abfb63c67c3b48b8b4c0bdec2f68d2b4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:06:12.461504 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:12.462833 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:12.501927 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbe17d-8648-4b36-85eb-2032b1e3d259 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:12.506651 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:12.507291 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.507566 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.507949 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.508573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:12.510514 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.511007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:12.529990 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55130d2b-da4a-47e3-bd5c-20655bbb577f admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:12.534759 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:12.535423 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.535704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.535982 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] PATH_INFO: `/v3/users/a6999b66901e49638e5979843209bc98` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.537120 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6999b66901e49638e5979843209bc98)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:12.539145 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.671086 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1397/2819] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:06:12 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:06:12.674940 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:12.677281 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:12.690100 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d4e9-c4c8-4855-9155-270274c0a98d admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:12.694570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3499b00-f27d-42f7-91c7-bc11b6c65dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:12.695182 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3499b00-f27d-42f7-91c7-bc11b6c65dfd admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.695453 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3499b00-f27d-42f7-91c7-bc11b6c65dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.695719 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3499b00-f27d-42f7-91c7-bc11b6c65dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.696372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3499b00-f27d-42f7-91c7-bc11b6c65dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:12.697976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3499b00-f27d-42f7-91c7-bc11b6c65dfd admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.708296 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1398/2820] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:06:12.717192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3499b00-f27d-42f7-91c7-bc11b6c65dfd admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:12.726540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3499b00-f27d-42f7-91c7-bc11b6c65dfd admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:12.731196 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b36e-058c-4823-80a4-ac463e3caab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:12.731746 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b36e-058c-4823-80a4-ac463e3caab7 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.732067 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b36e-058c-4823-80a4-ac463e3caab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.732337 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b36e-058c-4823-80a4-ac463e3caab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.732958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29b36e-058c-4823-80a4-ac463e3caab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:12.734469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29b36e-058c-4823-80a4-ac463e3caab7 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.744515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1399/2821] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 16:06:12.748741 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b36e-058c-4823-80a4-ac463e3caab7 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:12.758054 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b36e-058c-4823-80a4-ac463e3caab7 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:12.763349 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7271236-40b0-4335-89c0-a2b1f74826e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:12.763985 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7271236-40b0-4335-89c0-a2b1f74826e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.764259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7271236-40b0-4335-89c0-a2b1f74826e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.764528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7271236-40b0-4335-89c0-a2b1f74826e3 admin admin] PATH_INFO: `/v3/projects/58f26c0a7ebb42a39d2eefe9eab38634/users/db8572fee3f94cdb9faea5854101afc3/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.778678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7271236-40b0-4335-89c0-a2b1f74826e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f26c0a7ebb42a39d2eefe9eab38634, user_id=db8572fee3f94cdb9faea5854101afc3, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:12.781311 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7271236-40b0-4335-89c0-a2b1f74826e3 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:12.810101 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6999b66901e49638e5979843209bc98'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.810775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6999b66901e49638e5979843209bc98'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.822603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6999b66901e49638e5979843209bc98'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.877869 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1400/2822] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:06:12 2022] PUT /identity/v3/projects/58f26c0a7ebb42a39d2eefe9eab38634/users/db8572fee3f94cdb9faea5854101afc3/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:06:12.889573 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5f35-6a2c-4e2c-a20f-0bc4e477ee59 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:12.889855 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5f35-6a2c-4e2c-a20f-0bc4e477ee59 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:12.890126 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5f35-6a2c-4e2c-a20f-0bc4e477ee59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:12.917128 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-5f6d5f35-6a2c-4e2c-a20f-0bc4e477ee59 None None] MFA Rules not processed for user `db8572fee3f94cdb9faea5854101afc3`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:12.918423 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5f6d5f35-6a2c-4e2c-a20f-0bc4e477ee59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:12.957351 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6999b66901e49638e5979843209bc98'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.961334 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6999b66901e49638e5979843209bc98'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:12.968459 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1423/2823] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:12 2022] DELETE /identity/v3/users/a6999b66901e49638e5979843209bc98 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:06:12.990188 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1401/2824] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:06:13.007301 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:13.008909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:13.071060 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e492dac-6bd9-47af-ae84-c848d2ccb114 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:13.074890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc793543-4865-4a75-81bb-f6adee1c8946 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23ddf8f600454da7ae9b715cd4cbd186', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:13.075712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc793543-4865-4a75-81bb-f6adee1c8946 service glance] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:13.075983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc793543-4865-4a75-81bb-f6adee1c8946 service glance] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:13.076251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc793543-4865-4a75-81bb-f6adee1c8946 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:13.135280 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc793543-4865-4a75-81bb-f6adee1c8946 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:13.136738 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc793543-4865-4a75-81bb-f6adee1c8946 service glance] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:13.146149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1424/2825] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:13.439840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5f35-6a2c-4e2c-a20f-0bc4e477ee59 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:13.441467 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-5f6d5f35-6a2c-4e2c-a20f-0bc4e477ee59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:13.487689 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:13.513790 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5f35-6a2c-4e2c-a20f-0bc4e477ee59 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:13.571767 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:13.572393 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:13.576450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:13.576450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] PATH_INFO: `/v3/projects/c7165d3f6d014b528e97a07a2056c0b0` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:13.576450 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7165d3f6d014b528e97a07a2056c0b0)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:13.576770 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:13.722088 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7165d3f6d014b528e97a07a2056c0b0', 'request_id': 'req-4015c56d-cf52-4fdf-ab74-2af30535b9f5'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:13.727613 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7165d3f6d014b528e97a07a2056c0b0', 'request_id': 'req-4015c56d-cf52-4fdf-ab74-2af30535b9f5'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:13.819451 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7165d3f6d014b528e97a07a2056c0b0', 'request_id': 'req-4015c56d-cf52-4fdf-ab74-2af30535b9f5'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:13.831444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] The token cache is being invalidate because project c7165d3f6d014b528e97a07a2056c0b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:13.831732 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:13.832361 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:13.833847 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1402/2826] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:13 2022] DELETE /identity/v3/projects/c7165d3f6d014b528e97a07a2056c0b0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:13.837885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc793543-4865-4a75-81bb-f6adee1c8946 service glance] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:13.849249 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc793543-4865-4a75-81bb-f6adee1c8946 service glance] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:13.864086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:13.909558 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:13.910192 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:13.910470 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:13.910737 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] PATH_INFO: `/v3/users/97221877843546009e15c96ab204e4fd` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:13.911885 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97221877843546009e15c96ab204e4fd)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:13.913794 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:14.187308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97221877843546009e15c96ab204e4fd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:14.187959 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97221877843546009e15c96ab204e4fd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:14.200452 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97221877843546009e15c96ab204e4fd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:14.259415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97221877843546009e15c96ab204e4fd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:14.267550 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-25a08970-c119-4a33-b2c3-992c964ea4c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97221877843546009e15c96ab204e4fd'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:14.273514 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1425/2827] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:13 2022] DELETE /identity/v3/users/97221877843546009e15c96ab204e4fd => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:14.712626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:14.714688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:14.728657 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015c56d-cf52-4fdf-ab74-2af30535b9f5 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:14.790481 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:14.791138 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:14.791414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:14.791684 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] PATH_INFO: `/v3/projects/9ffa8acac21b46b48f92da6823f16e89` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:14.792787 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffa8acac21b46b48f92da6823f16e89)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:14.795440 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:14.813413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:14.942437 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffa8acac21b46b48f92da6823f16e89', 'request_id': 'req-d4b35a8a-3c22-43d8-a001-c9044d68f489'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:14.948037 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffa8acac21b46b48f92da6823f16e89', 'request_id': 'req-d4b35a8a-3c22-43d8-a001-c9044d68f489'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:15.019375 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffa8acac21b46b48f92da6823f16e89', 'request_id': 'req-d4b35a8a-3c22-43d8-a001-c9044d68f489'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:15.032697 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] The token cache is being invalidate because project 9ffa8acac21b46b48f92da6823f16e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:15.033051 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:15.033823 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-d4b35a8a-3c22-43d8-a001-c9044d68f489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:15.035469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1403/2828] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:14 2022] DELETE /identity/v3/projects/9ffa8acac21b46b48f92da6823f16e89 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:15.040930 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b338e-a284-46f5-8e63-b172fddc9e96 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.041535 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b338e-a284-46f5-8e63-b172fddc9e96 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.041864 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b338e-a284-46f5-8e63-b172fddc9e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.067443 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-cd6b338e-a284-46f5-8e63-b172fddc9e96 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:15.068701 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-cd6b338e-a284-46f5-8e63-b172fddc9e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:15.129449 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1426/2829] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:15.137428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055a3ab-0ca8-4647-8aa3-6dca67241d8f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.137769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055a3ab-0ca8-4647-8aa3-6dca67241d8f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.138050 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055a3ab-0ca8-4647-8aa3-6dca67241d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.159655 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-f055a3ab-0ca8-4647-8aa3-6dca67241d8f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:15.160907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f055a3ab-0ca8-4647-8aa3-6dca67241d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:15.168870 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1404/2830] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:15.174174 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b338e-a284-46f5-8e63-b172fddc9e96 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:15.184053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b338e-a284-46f5-8e63-b172fddc9e96 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:15.189182 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:15.189777 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.190051 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.190319 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.190988 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:06:15.191286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:15.192883 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:15.204920 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1427/2831] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:06:15.209036 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055a3ab-0ca8-4647-8aa3-6dca67241d8f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:15.219166 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055a3ab-0ca8-4647-8aa3-6dca67241d8f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:15.224033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8c7e8-80de-4732-9f03-c08215624d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:15.224645 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8c7e8-80de-4732-9f03-c08215624d67 admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.224918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8c7e8-80de-4732-9f03-c08215624d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.225186 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8c7e8-80de-4732-9f03-c08215624d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.225821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8c7e8-80de-4732-9f03-c08215624d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:15.227757 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8c7e8-80de-4732-9f03-c08215624d67 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:15.318848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1405/2832] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:06:15 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:15.322942 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:15.333651 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e94a4-158d-407a-b810-45df8c0e2925 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:15.354528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:15.391665 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa372c89-8aa1-460f-b2fa-1efbcb8558a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:15.392271 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372c89-8aa1-460f-b2fa-1efbcb8558a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.392547 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372c89-8aa1-460f-b2fa-1efbcb8558a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.392816 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa372c89-8aa1-460f-b2fa-1efbcb8558a3 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.393455 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa372c89-8aa1-460f-b2fa-1efbcb8558a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:15.395442 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa372c89-8aa1-460f-b2fa-1efbcb8558a3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:15.545910 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1428/2833] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:06:15 2022] POST /identity/v3/users => generated 282 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 16:06:15.549953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8c7e8-80de-4732-9f03-c08215624d67 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:15.561821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8c7e8-80de-4732-9f03-c08215624d67 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:15.566755 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ab38c-b266-4eb7-9cdd-3197581d247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:15.567391 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab38c-b266-4eb7-9cdd-3197581d247b admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.567663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab38c-b266-4eb7-9cdd-3197581d247b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.567926 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ab38c-b266-4eb7-9cdd-3197581d247b admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.568560 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ab38c-b266-4eb7-9cdd-3197581d247b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:15.570302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ab38c-b266-4eb7-9cdd-3197581d247b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:15.581622 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1406/2834] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:15.585674 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa372c89-8aa1-460f-b2fa-1efbcb8558a3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:15.596086 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa372c89-8aa1-460f-b2fa-1efbcb8558a3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:15.600822 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b5aa8-0310-4dbd-8e3f-a7cf6736f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:15.601412 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b5aa8-0310-4dbd-8e3f-a7cf6736f6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.601691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b5aa8-0310-4dbd-8e3f-a7cf6736f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.601963 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b5aa8-0310-4dbd-8e3f-a7cf6736f6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.602631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b5aa8-0310-4dbd-8e3f-a7cf6736f6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:15.604333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b5aa8-0310-4dbd-8e3f-a7cf6736f6f9 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:15.615972 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1429/2835] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:15.620261 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ab38c-b266-4eb7-9cdd-3197581d247b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:15.631246 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ab38c-b266-4eb7-9cdd-3197581d247b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:15.635715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:15.636352 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.636625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.636890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] PATH_INFO: `/v3/projects/c4e06c34ac9a4370b547d0d4a310314a/users/d62ee60f0c1f46778d1897c87decf144/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.651944 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e06c34ac9a4370b547d0d4a310314a, user_id=d62ee60f0c1f46778d1897c87decf144, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:15.654729 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:15.745392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1407/2836] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:06:15 2022] PUT /identity/v3/projects/c4e06c34ac9a4370b547d0d4a310314a/users/d62ee60f0c1f46778d1897c87decf144/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:15.756699 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2588e-3b64-44cd-96de-c96685ee4833 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.757031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2588e-3b64-44cd-96de-c96685ee4833 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.757309 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2588e-3b64-44cd-96de-c96685ee4833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.790251 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-5ab2588e-3b64-44cd-96de-c96685ee4833 None None] MFA Rules not processed for user `d62ee60f0c1f46778d1897c87decf144`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:15.791518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5ab2588e-3b64-44cd-96de-c96685ee4833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:15.859886 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1430/2837] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:15.871214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:15.872889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:15.937947 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a05249-1493-429f-a0b8-d6fffc14d26a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:15.942736 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925510-f702-4291-a395-5215ee337bb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:15.943473 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925510-f702-4291-a395-5215ee337bb4 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:15.943890 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925510-f702-4291-a395-5215ee337bb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:15.944159 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925510-f702-4291-a395-5215ee337bb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:15.953643 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925510-f702-4291-a395-5215ee337bb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:15.955214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925510-f702-4291-a395-5215ee337bb4 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:15.963957 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1408/2838] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:25.856928 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:25.959967 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:36.919796 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2588e-3b64-44cd-96de-c96685ee4833 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:36.921186 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5ab2588e-3b64-44cd-96de-c96685ee4833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:36.989112 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2588e-3b64-44cd-96de-c96685ee4833 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:36.994397 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:36.995046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:36.995320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:36.995586 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] PATH_INFO: `/v3/users/db8572fee3f94cdb9faea5854101afc3` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:36.996704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8572fee3f94cdb9faea5854101afc3)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:36.998623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:37.152957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8572fee3f94cdb9faea5854101afc3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:37.153528 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8572fee3f94cdb9faea5854101afc3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:37.166482 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8572fee3f94cdb9faea5854101afc3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:37.258867 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8572fee3f94cdb9faea5854101afc3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:37.266831 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8572fee3f94cdb9faea5854101afc3'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:37.275499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1431/2839] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:36 2022] DELETE /identity/v3/users/db8572fee3f94cdb9faea5854101afc3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:37.280085 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92aec-487d-4463-a4db-54ab2109ce44 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:37.280599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92aec-487d-4463-a4db-54ab2109ce44 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:37.280705 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d92aec-487d-4463-a4db-54ab2109ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:37.300357 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-36d92aec-487d-4463-a4db-54ab2109ce44 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:37.301677 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-36d92aec-487d-4463-a4db-54ab2109ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:37.366676 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1409/2840] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:37.376733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:37.378165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:37.439807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69125f-58f2-4c58-9e79-8c913d5dfdf0 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:37.442588 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff68224-27c5-4b6b-a785-39b5d8c8870b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:37.443853 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff68224-27c5-4b6b-a785-39b5d8c8870b service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:37.444127 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff68224-27c5-4b6b-a785-39b5d8c8870b service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:37.444398 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff68224-27c5-4b6b-a785-39b5d8c8870b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:37.453533 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff68224-27c5-4b6b-a785-39b5d8c8870b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:37.455059 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff68224-27c5-4b6b-a785-39b5d8c8870b service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:37.464955 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1432/2841] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:37.952156 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d92aec-487d-4463-a4db-54ab2109ce44 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:37.953637 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-36d92aec-487d-4463-a4db-54ab2109ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:37.971978 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d92aec-487d-4463-a4db-54ab2109ce44 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:37.979055 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:37.979710 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:37.979986 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:37.980257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] PATH_INFO: `/v3/projects/58f26c0a7ebb42a39d2eefe9eab38634` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:37.981396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f26c0a7ebb42a39d2eefe9eab38634)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:37.983448 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:38.090892 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f26c0a7ebb42a39d2eefe9eab38634', 'request_id': 'req-3fd97354-95c4-45df-a029-e1cc60d35f41'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:38.096567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f26c0a7ebb42a39d2eefe9eab38634', 'request_id': 'req-3fd97354-95c4-45df-a029-e1cc60d35f41'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:38.173113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f26c0a7ebb42a39d2eefe9eab38634', 'request_id': 'req-3fd97354-95c4-45df-a029-e1cc60d35f41'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:38.185803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] The token cache is being invalidate because project 58f26c0a7ebb42a39d2eefe9eab38634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:38.186091 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:38.186773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-3fd97354-95c4-45df-a029-e1cc60d35f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:38.188570 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1410/2842] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:37 2022] DELETE /identity/v3/projects/58f26c0a7ebb42a39d2eefe9eab38634 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:38.193466 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c774003-72ac-495e-8ee6-db4b5e4a0db6 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.193929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c774003-72ac-495e-8ee6-db4b5e4a0db6 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.194266 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c774003-72ac-495e-8ee6-db4b5e4a0db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.214154 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-1c774003-72ac-495e-8ee6-db4b5e4a0db6 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:38.215392 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-1c774003-72ac-495e-8ee6-db4b5e4a0db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:38.233024 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:38.287087 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1433/2843] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:38.347716 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3e00-895a-4b89-a748-f39eec8b4a75 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.348305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3e00-895a-4b89-a748-f39eec8b4a75 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.348305 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3e00-895a-4b89-a748-f39eec8b4a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.368060 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c26c3e00-895a-4b89-a748-f39eec8b4a75 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:38.369283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c26c3e00-895a-4b89-a748-f39eec8b4a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:38.380972 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1411/2844] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:38.385124 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c774003-72ac-495e-8ee6-db4b5e4a0db6 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:38.395100 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c774003-72ac-495e-8ee6-db4b5e4a0db6 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:38.400478 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:38.401079 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.401355 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.401622 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.402384 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88508) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:06:38.402685 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:38.404341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:38.416102 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1434/2845] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:06:38.420461 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c3e00-895a-4b89-a748-f39eec8b4a75 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:38.429799 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c3e00-895a-4b89-a748-f39eec8b4a75 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:38.434513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:38.435148 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.435428 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.435695 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.436310 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:38.438469 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:38.578593 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1412/2846] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:06:38 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:38.581603 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:38.591944 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9bcae-2a2a-4062-bf8a-428a1f3db6e3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:38.650511 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ac8e6-9e76-4650-85a1-1b9f239819b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:38.651190 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac8e6-9e76-4650-85a1-1b9f239819b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.651727 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac8e6-9e76-4650-85a1-1b9f239819b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.652005 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac8e6-9e76-4650-85a1-1b9f239819b1 admin admin] PATH_INFO: `/v3/users` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.652623 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ac8e6-9e76-4650-85a1-1b9f239819b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:38.654987 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ac8e6-9e76-4650-85a1-1b9f239819b1 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:38.694598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:38.696324 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:38.749388 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:38.754620 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef4499-c418-477b-966d-5b25eec24b9c admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:38.758381 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:38.759007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.759284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.759553 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.761325 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:38.774584 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1435/2847] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:06:38 2022] POST /identity/v3/users => generated 281 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:38.779525 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ac8e6-9e76-4650-85a1-1b9f239819b1 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:38.793557 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ac8e6-9e76-4650-85a1-1b9f239819b1 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:38.798427 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe1be1-95fc-42de-b4b0-207338946d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:38.799075 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe1be1-95fc-42de-b4b0-207338946d24 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.799358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe1be1-95fc-42de-b4b0-207338946d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.799631 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe1be1-95fc-42de-b4b0-207338946d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.800268 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe1be1-95fc-42de-b4b0-207338946d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:38.801859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe1be1-95fc-42de-b4b0-207338946d24 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:38.813734 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1436/2848] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:38.818119 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe1be1-95fc-42de-b4b0-207338946d24 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:38.828242 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe1be1-95fc-42de-b4b0-207338946d24 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:38.829528 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:38.830910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:38.832926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f86b4c-41d8-4488-9a59-b1863fd81f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:38.834162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f86b4c-41d8-4488-9a59-b1863fd81f49 admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.834641 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f86b4c-41d8-4488-9a59-b1863fd81f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.835056 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f86b4c-41d8-4488-9a59-b1863fd81f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.835726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f86b4c-41d8-4488-9a59-b1863fd81f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:38.837353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f86b4c-41d8-4488-9a59-b1863fd81f49 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:38.841192 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1413/2849] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:06:38.851113 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1437/2850] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:38.855277 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:38.866475 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aff21b-8e73-40dd-9336-0cdb1c26d319 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:38.875536 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:38.879769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:38.880158 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:38.880548 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] PATH_INFO: `/v3/projects/c34bb6669cc64292b4b67d7b64aa2343/users/2591a9d0cbb947f29d49d9a8d36c5cb5/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:38.899427 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c34bb6669cc64292b4b67d7b64aa2343, user_id=2591a9d0cbb947f29d49d9a8d36c5cb5, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:38.902125 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:38.991531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1414/2851] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:06:38 2022] PUT /identity/v3/projects/c34bb6669cc64292b4b67d7b64aa2343/users/2591a9d0cbb947f29d49d9a8d36c5cb5/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:06:39.002152 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17ecd3-c898-40b7-8bb8-26d671cf50a1 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:39.002468 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17ecd3-c898-40b7-8bb8-26d671cf50a1 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:39.002859 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17ecd3-c898-40b7-8bb8-26d671cf50a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:39.036681 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-ce17ecd3-c898-40b7-8bb8-26d671cf50a1 None None] MFA Rules not processed for user `2591a9d0cbb947f29d49d9a8d36c5cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:39.037929 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-ce17ecd3-c898-40b7-8bb8-26d671cf50a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:39.115110 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1438/2852] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 16:06:39.145501 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:39.148040 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:39.211033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09ccaf-2497-4550-bbd3-42e3a6eb7c16 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:39.215147 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbfc21-d3fd-47c6-a281-4aae46613ba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:39.217049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfc21-d3fd-47c6-a281-4aae46613ba6 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:39.217049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfc21-d3fd-47c6-a281-4aae46613ba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:39.217049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfc21-d3fd-47c6-a281-4aae46613ba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:39.225953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbfc21-d3fd-47c6-a281-4aae46613ba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:39.227422 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbfc21-d3fd-47c6-a281-4aae46613ba6 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:39.236889 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1415/2853] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:39.252380 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17ecd3-c898-40b7-8bb8-26d671cf50a1 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:39.261636 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:39.326043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17ecd3-c898-40b7-8bb8-26d671cf50a1 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:39.330691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:39.331467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:39.331716 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:39.332010 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] PATH_INFO: `/v3/users/2591a9d0cbb947f29d49d9a8d36c5cb5` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:39.333134 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2591a9d0cbb947f29d49d9a8d36c5cb5)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:39.335658 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:39.526232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2591a9d0cbb947f29d49d9a8d36c5cb5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:39.526890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2591a9d0cbb947f29d49d9a8d36c5cb5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:39.539815 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2591a9d0cbb947f29d49d9a8d36c5cb5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:39.636147 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2591a9d0cbb947f29d49d9a8d36c5cb5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:39.643726 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2591a9d0cbb947f29d49d9a8d36c5cb5'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:39.650269 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1439/2854] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:39 2022] DELETE /identity/v3/users/2591a9d0cbb947f29d49d9a8d36c5cb5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:39.654858 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a763d8-f97e-4975-baf4-705a71228a39 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:39.655406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a763d8-f97e-4975-baf4-705a71228a39 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:39.655406 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a763d8-f97e-4975-baf4-705a71228a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:39.675791 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-92a763d8-f97e-4975-baf4-705a71228a39 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:39.677052 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-92a763d8-f97e-4975-baf4-705a71228a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:39.741908 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1416/2855] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:39.752054 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:39.753498 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:39.823046 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0313-1d28-41ed-b763-22a34c15e166 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:39.830880 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:39.832167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:39.832598 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:39.833007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:39.848718 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:39.850538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:39.863731 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1440/2856] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:06:40.338379 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a763d8-f97e-4975-baf4-705a71228a39 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:40.339994 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-92a763d8-f97e-4975-baf4-705a71228a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:40.367583 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a763d8-f97e-4975-baf4-705a71228a39 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:40.375704 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f7db5-9edd-4a81-aa17-df719655f7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:40.376510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f7db5-9edd-4a81-aa17-df719655f7ea service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:40.381914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f7db5-9edd-4a81-aa17-df719655f7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:40.381914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f7db5-9edd-4a81-aa17-df719655f7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:40.381914 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f42f7db5-9edd-4a81-aa17-df719655f7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:40.400971 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:40.473942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f7db5-9edd-4a81-aa17-df719655f7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:40.473942 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f7db5-9edd-4a81-aa17-df719655f7ea service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:40.474410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:40.474410 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:40.485803 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1417/2857] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:40.509123 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2364ba-0e12-46eb-9089-8b5c5c82f0c8 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:40.514602 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:40.515282 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:40.515608 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:40.515876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] PATH_INFO: `/v3/projects/c34bb6669cc64292b4b67d7b64aa2343` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:40.517014 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=c34bb6669cc64292b4b67d7b64aa2343)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:40.518952 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:40.550769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:40.687581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c34bb6669cc64292b4b67d7b64aa2343', 'request_id': 'req-677cebcf-ca9d-4920-96f5-efef1661a1ad'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:40.694430 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c34bb6669cc64292b4b67d7b64aa2343', 'request_id': 'req-677cebcf-ca9d-4920-96f5-efef1661a1ad'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:40.841543 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c34bb6669cc64292b4b67d7b64aa2343', 'request_id': 'req-677cebcf-ca9d-4920-96f5-efef1661a1ad'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:40.854352 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] The token cache is being invalidate because project c34bb6669cc64292b4b67d7b64aa2343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88508) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:40.861413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:40.861413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-677cebcf-ca9d-4920-96f5-efef1661a1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:40.861413 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1441/2858] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:40 2022] DELETE /identity/v3/projects/c34bb6669cc64292b4b67d7b64aa2343 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:40.863260 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c91bdd-e90e-4c3c-b5f4-6ca90671656f None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:40.863726 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c91bdd-e90e-4c3c-b5f4-6ca90671656f None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:40.863830 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c91bdd-e90e-4c3c-b5f4-6ca90671656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:40.892963 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-99c91bdd-e90e-4c3c-b5f4-6ca90671656f None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:40.894219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-99c91bdd-e90e-4c3c-b5f4-6ca90671656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:40.963435 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1418/2859] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:40.971365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9054e2-a187-4fc3-9563-86ee567c7d41 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:40.983940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9054e2-a187-4fc3-9563-86ee567c7d41 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:40.983940 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9054e2-a187-4fc3-9563-86ee567c7d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:40.993071 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-4b9054e2-a187-4fc3-9563-86ee567c7d41 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:40.994421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-4b9054e2-a187-4fc3-9563-86ee567c7d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:41.004065 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1442/2860] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:41.008537 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c91bdd-e90e-4c3c-b5f4-6ca90671656f None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.018580 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c91bdd-e90e-4c3c-b5f4-6ca90671656f None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.023915 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.024517 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.024796 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.025065 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.025773 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88509) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 16:06:41.026074 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.027723 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:41.040116 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1419/2861] 10.222.0.94 () {58 vars in 1153 bytes} [Fri Aug 26 16:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 16:06:41.044769 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9054e2-a187-4fc3-9563-86ee567c7d41 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.054073 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9054e2-a187-4fc3-9563-86ee567c7d41 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.058704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a84cf-297a-4ca7-955f-e442272d60ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.059333 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a84cf-297a-4ca7-955f-e442272d60ab admin admin] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.059606 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a84cf-297a-4ca7-955f-e442272d60ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.059871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a84cf-297a-4ca7-955f-e442272d60ab admin admin] PATH_INFO: `/v3/projects` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.060489 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a84cf-297a-4ca7-955f-e442272d60ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.062383 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a84cf-297a-4ca7-955f-e442272d60ab admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:41.181246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1443/2862] 10.222.0.94 () {60 vars in 1153 bytes} [Fri Aug 26 16:06:41 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:41.185368 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.196456 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5622a8-e593-47e8-92cb-2c5a57465f25 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.253598 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff223262-3630-4d22-beb3-f447999f454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.254198 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff223262-3630-4d22-beb3-f447999f454a admin admin] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.254468 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff223262-3630-4d22-beb3-f447999f454a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.254731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff223262-3630-4d22-beb3-f447999f454a admin admin] PATH_INFO: `/v3/users` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.256132 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff223262-3630-4d22-beb3-f447999f454a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.258033 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff223262-3630-4d22-beb3-f447999f454a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:41.339421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1420/2863] 10.222.0.94 () {60 vars in 1144 bytes} [Fri Aug 26 16:06:41 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 16:06:41.343991 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a84cf-297a-4ca7-955f-e442272d60ab admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.354948 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a84cf-297a-4ca7-955f-e442272d60ab admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.359926 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539ea9f-38b3-4dc4-bb5f-18337b3c50dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.360523 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ea9f-38b3-4dc4-bb5f-18337b3c50dd admin admin] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.360801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ea9f-38b3-4dc4-bb5f-18337b3c50dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.361160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ea9f-38b3-4dc4-bb5f-18337b3c50dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.361829 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539ea9f-38b3-4dc4-bb5f-18337b3c50dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.363460 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539ea9f-38b3-4dc4-bb5f-18337b3c50dd admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:41.374440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1444/2864] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:41.378217 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff223262-3630-4d22-beb3-f447999f454a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.388680 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff223262-3630-4d22-beb3-f447999f454a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.393380 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbcff0-14d7-4a2f-af86-714ef7c34074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.393976 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbcff0-14d7-4a2f-af86-714ef7c34074 admin admin] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.394259 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbcff0-14d7-4a2f-af86-714ef7c34074 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.394531 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbcff0-14d7-4a2f-af86-714ef7c34074 admin admin] PATH_INFO: `/v3/roles` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.395212 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbcff0-14d7-4a2f-af86-714ef7c34074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.396923 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbcff0-14d7-4a2f-af86-714ef7c34074 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:41.408520 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1421/2865] 10.222.0.94 () {58 vars in 1122 bytes} [Fri Aug 26 16:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 16:06:41.413687 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539ea9f-38b3-4dc4-bb5f-18337b3c50dd admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.426042 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539ea9f-38b3-4dc4-bb5f-18337b3c50dd admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.431326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.431924 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.432196 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.432458 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] PATH_INFO: `/v3/projects/5aba6050ca6f4610aec87ac559c4ec8f/users/350862f657aa4a9387688e3b6a77c0b6/roles/95375b0480bf4855a3ce51d1985c9b57` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.448440 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aba6050ca6f4610aec87ac559c4ec8f, user_id=350862f657aa4a9387688e3b6a77c0b6, role_id=95375b0480bf4855a3ce51d1985c9b57)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.451193 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:41.530567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1445/2866] 10.222.0.94 () {60 vars in 1483 bytes} [Fri Aug 26 16:06:41 2022] PUT /identity/v3/projects/5aba6050ca6f4610aec87ac559c4ec8f/users/350862f657aa4a9387688e3b6a77c0b6/roles/95375b0480bf4855a3ce51d1985c9b57 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:41.543149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be96e1a-b86e-4cb9-a8eb-97515c3ece5a None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.543431 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be96e1a-b86e-4cb9-a8eb-97515c3ece5a None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.543714 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be96e1a-b86e-4cb9-a8eb-97515c3ece5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.569413 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:41.578688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-7be96e1a-b86e-4cb9-a8eb-97515c3ece5a None None] MFA Rules not processed for user `350862f657aa4a9387688e3b6a77c0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:41.584476 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7be96e1a-b86e-4cb9-a8eb-97515c3ece5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:41.648860 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1422/2867] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:41.661747 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.663678 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:41.698769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be96e1a-b86e-4cb9-a8eb-97515c3ece5a None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.700139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-7be96e1a-b86e-4cb9-a8eb-97515c3ece5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:41.727743 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576d21-a593-42d6-82ed-11904dd27f70 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.731401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7fe5-ce94-4358-ad7a-6f7a7ecb687e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.732000 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7fe5-ce94-4358-ad7a-6f7a7ecb687e service cinder] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.732274 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7fe5-ce94-4358-ad7a-6f7a7ecb687e service cinder] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.732648 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef7fe5-ce94-4358-ad7a-6f7a7ecb687e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.743178 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef7fe5-ce94-4358-ad7a-6f7a7ecb687e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.744977 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef7fe5-ce94-4358-ad7a-6f7a7ecb687e service cinder] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:41.755733 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1446/2868] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:41.761845 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be96e1a-b86e-4cb9-a8eb-97515c3ece5a None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.765508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18069378-d9b5-46d6-9784-d4195571aa28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.766139 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069378-d9b5-46d6-9784-d4195571aa28 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.766414 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069378-d9b5-46d6-9784-d4195571aa28 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.766685 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069378-d9b5-46d6-9784-d4195571aa28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.768489 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-18069378-d9b5-46d6-9784-d4195571aa28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:41.783748 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7fe5-ce94-4358-ad7a-6f7a7ecb687e service cinder] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:41.850499 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18069378-d9b5-46d6-9784-d4195571aa28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.852497 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18069378-d9b5-46d6-9784-d4195571aa28 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:41.858520 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:41.870133 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1423/2869] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 16:06:41.884844 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef7fe5-ce94-4358-ad7a-6f7a7ecb687e service cinder] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:41.889909 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:41.890554 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:41.890828 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:41.892611 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] PATH_INFO: `/v3/users/350862f657aa4a9387688e3b6a77c0b6` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:41.893848 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] RBAC: Authorizing `identity:delete_user(user_id=350862f657aa4a9387688e3b6a77c0b6)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:41.896485 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:42.095813 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '350862f657aa4a9387688e3b6a77c0b6'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:42.096491 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '350862f657aa4a9387688e3b6a77c0b6'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:42.111996 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '350862f657aa4a9387688e3b6a77c0b6'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:42.218861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '350862f657aa4a9387688e3b6a77c0b6'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:42.227141 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '350862f657aa4a9387688e3b6a77c0b6'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:42.234286 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1447/2870] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:41 2022] DELETE /identity/v3/users/350862f657aa4a9387688e3b6a77c0b6 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:42.238706 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa304b6-5a5c-491f-8311-544db53c9038 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:42.239022 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa304b6-5a5c-491f-8311-544db53c9038 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:42.239319 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa304b6-5a5c-491f-8311-544db53c9038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:42.261018 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-9aa304b6-5a5c-491f-8311-544db53c9038 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:42.263208 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9aa304b6-5a5c-491f-8311-544db53c9038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:42.314533 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=100 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:42.340309 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1424/2871] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:42.352254 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:42.353704 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:42.429018 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38092b5a-8df0-4986-bab5-064aac5e78da admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:42.434966 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3fd6f-0c00-4978-b35b-ff9eb440764b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:42.435232 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3fd6f-0c00-4978-b35b-ff9eb440764b service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:42.435506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3fd6f-0c00-4978-b35b-ff9eb440764b service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:42.435775 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3fd6f-0c00-4978-b35b-ff9eb440764b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:42.452691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3fd6f-0c00-4978-b35b-ff9eb440764b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:42.457888 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3fd6f-0c00-4978-b35b-ff9eb440764b service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:42.467696 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1448/2872] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:43.409286 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa304b6-5a5c-491f-8311-544db53c9038 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:43.410814 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-9aa304b6-5a5c-491f-8311-544db53c9038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:43.433119 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa304b6-5a5c-491f-8311-544db53c9038 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:43.452209 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:43.452859 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:43.453149 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:43.453417 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] PATH_INFO: `/v3/projects/5aba6050ca6f4610aec87ac559c4ec8f` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:43.454555 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aba6050ca6f4610aec87ac559c4ec8f)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:43.456512 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:43.631715 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aba6050ca6f4610aec87ac559c4ec8f', 'request_id': 'req-ae84555a-e906-4caa-9aa3-fca65b1b837a'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:43.637256 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aba6050ca6f4610aec87ac559c4ec8f', 'request_id': 'req-ae84555a-e906-4caa-9aa3-fca65b1b837a'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:43.767236 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aba6050ca6f4610aec87ac559c4ec8f', 'request_id': 'req-ae84555a-e906-4caa-9aa3-fca65b1b837a'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:43.780656 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] The token cache is being invalidate because project 5aba6050ca6f4610aec87ac559c4ec8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:43.780953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:43.781658 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:43.783727 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1425/2873] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:43 2022] DELETE /identity/v3/projects/5aba6050ca6f4610aec87ac559c4ec8f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:45.804958 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3fd6f-0c00-4978-b35b-ff9eb440764b service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:45.814253 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3fd6f-0c00-4978-b35b-ff9eb440764b service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:45.866784 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:45.867431 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:45.867708 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:45.867979 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:45.869758 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:45.896175 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:45.933171 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:45.934803 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:45.952283 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1449/2874] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:49.453082 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:49.454510 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:49.527818 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84555a-e906-4caa-9aa3-fca65b1b837a admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:49.533049 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:49.533654 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:49.533922 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:49.536678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:49.536678 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:49.547919 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:49.549515 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:49.560284 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1426/2875] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:53.633093 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:53.634601 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:53.648599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20402e-06d2-4b3a-9f3c-00179d6e3fbe service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:53.654188 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:53.654847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:53.655613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:53.655613 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] PATH_INFO: `/v3/users/3b3abcfcc7c6408994e27b7c9937c2be` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:53.656740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3abcfcc7c6408994e27b7c9937c2be)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:53.659036 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:53.849740 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3abcfcc7c6408994e27b7c9937c2be'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:53.850341 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3abcfcc7c6408994e27b7c9937c2be'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:53.862358 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3abcfcc7c6408994e27b7c9937c2be'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:53.944691 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3abcfcc7c6408994e27b7c9937c2be'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:53.952505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3abcfcc7c6408994e27b7c9937c2be'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:53.959429 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1450/2876] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:53 2022] DELETE /identity/v3/users/3b3abcfcc7c6408994e27b7c9937c2be => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:54.209203 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:54.222495 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fa233-5dbb-4f50-9b4f-d9c1c8d7297c service placement] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:54.274804 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:54.281513 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:54.282143 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:54.282421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:54.282694 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] PATH_INFO: `/v3/projects/ea82b722416a415393ff78428a277a01` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:54.283843 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea82b722416a415393ff78428a277a01)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:54.285818 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:54.506719 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea82b722416a415393ff78428a277a01', 'request_id': 'req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:54.512275 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea82b722416a415393ff78428a277a01', 'request_id': 'req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:54.607829 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea82b722416a415393ff78428a277a01', 'request_id': 'req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:54.620372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] The token cache is being invalidate because project ea82b722416a415393ff78428a277a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:54.620721 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:54.621372 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:54.623045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1427/2877] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:54 2022] DELETE /identity/v3/projects/ea82b722416a415393ff78428a277a01 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:54.627125 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:54.628552 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:54.642447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2e806-7ffe-4fe7-a3fa-57c7465b2c89 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:54.699162 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:54.699792 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:54.700072 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:54.700366 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] PATH_INFO: `/v3/users/3f0ca12d53eb4c6b989c501a714ce2cb` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:54.701497 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0ca12d53eb4c6b989c501a714ce2cb)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:54.703445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:54.962128 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:54.970393 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0ca12d53eb4c6b989c501a714ce2cb'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:54.971103 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0ca12d53eb4c6b989c501a714ce2cb'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:54.982768 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0ca12d53eb4c6b989c501a714ce2cb'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:55.120998 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0ca12d53eb4c6b989c501a714ce2cb'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:55.128624 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0ca12d53eb4c6b989c501a714ce2cb'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:55.135876 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1451/2878] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:54 2022] DELETE /identity/v3/users/3f0ca12d53eb4c6b989c501a714ce2cb => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:55.380929 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:55.383257 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:55.397997 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbae9b-ee8e-4cfa-9d81-120ee4499d1b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:55.464434 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:55.465115 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:55.465390 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:55.465656 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] PATH_INFO: `/v3/projects/46893205b06941b0b0123571e6a67a52` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:55.466771 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46893205b06941b0b0123571e6a67a52)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:55.468895 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:55.613569 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46893205b06941b0b0123571e6a67a52', 'request_id': 'req-018e9650-8a73-4b6d-a250-b6e679b5fb68'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:55.620199 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46893205b06941b0b0123571e6a67a52', 'request_id': 'req-018e9650-8a73-4b6d-a250-b6e679b5fb68'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:55.703951 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46893205b06941b0b0123571e6a67a52', 'request_id': 'req-018e9650-8a73-4b6d-a250-b6e679b5fb68'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:55.716004 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] The token cache is being invalidate because project 46893205b06941b0b0123571e6a67a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:55.716290 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:55.716917 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-018e9650-8a73-4b6d-a250-b6e679b5fb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:55.718459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1428/2879] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:55 2022] DELETE /identity/v3/projects/46893205b06941b0b0123571e6a67a52 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:06:58.539654 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:58.541077 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:58.553680 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefba8c2-70ef-46f9-9856-d37040cfe905 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:58.613377 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:58.614191 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:58.614475 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:58.614744 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] PATH_INFO: `/v3/users/d62ee60f0c1f46778d1897c87decf144` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:58.615937 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62ee60f0c1f46778d1897c87decf144)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:58.618556 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:58.808914 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62ee60f0c1f46778d1897c87decf144'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:58.809529 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62ee60f0c1f46778d1897c87decf144'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:58.822585 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62ee60f0c1f46778d1897c87decf144'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:58.915273 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62ee60f0c1f46778d1897c87decf144'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:58.923494 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62ee60f0c1f46778d1897c87decf144'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:58.930167 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1452/2880] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:06:58 2022] DELETE /identity/v3/users/d62ee60f0c1f46778d1897c87decf144 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 16:06:58.934628 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8162b49-67c8-4350-86aa-08d192af65b2 None None] REQUEST_METHOD: `POST` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:58.935128 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8162b49-67c8-4350-86aa-08d192af65b2 None None] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:58.935253 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8162b49-67c8-4350-86aa-08d192af65b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:58.956425 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.auth.core [None req-c8162b49-67c8-4350-86aa-08d192af65b2 None None] MFA Rules not processed for user `5f7a209666c0478690d711f7d361c03b`. Rule list: `[]` (Enabled: `True`). {{(pid=88509) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:06:58.957625 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c8162b49-67c8-4350-86aa-08d192af65b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:58.967046 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:59.017343 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1429/2881] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:06:59.027943 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:59.029399 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:59.067445 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:06:59.089810 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736d5d3-4a3e-4b0e-ba57-b71c280cbca3 admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:59.093638 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56257ca-dfb5-4442-876f-2672670c2b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:59.094246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56257ca-dfb5-4442-876f-2672670c2b67 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:59.094519 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56257ca-dfb5-4442-876f-2672670c2b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:59.094785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56257ca-dfb5-4442-876f-2672670c2b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:59.103983 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56257ca-dfb5-4442-876f-2672670c2b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:59.106291 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56257ca-dfb5-4442-876f-2672670c2b67 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:59.115378 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1453/2882] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:06:59.533959 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8162b49-67c8-4350-86aa-08d192af65b2 None None] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:06:59.536283 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-c8162b49-67c8-4350-86aa-08d192af65b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:06:59.550505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8162b49-67c8-4350-86aa-08d192af65b2 None None] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:06:59.556403 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:06:59.557129 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:06:59.557478 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:06:59.557662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] PATH_INFO: `/v3/projects/c4e06c34ac9a4370b547d0d4a310314a` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:06:59.558775 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e06c34ac9a4370b547d0d4a310314a)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:06:59.560743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:06:59.681938 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e06c34ac9a4370b547d0d4a310314a', 'request_id': 'req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:59.687444 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e06c34ac9a4370b547d0d4a310314a', 'request_id': 'req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:59.772459 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e06c34ac9a4370b547d0d4a310314a', 'request_id': 'req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:59.784315 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] The token cache is being invalidate because project c4e06c34ac9a4370b547d0d4a310314a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:06:59.784600 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:59.785238 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:06:59.786738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1430/2883] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:06:59 2022] DELETE /identity/v3/projects/c4e06c34ac9a4370b547d0d4a310314a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:07:09.395630 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=11 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:07:09.781254 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:07:42.067882 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56257ca-dfb5-4442-876f-2672670c2b67 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:07:42.076861 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56257ca-dfb5-4442-876f-2672670c2b67 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:07:42.127538 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb793c0-3d05-4345-9d5c-0251fbcc3000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:07:42.128158 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb793c0-3d05-4345-9d5c-0251fbcc3000 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:07:42.128434 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb793c0-3d05-4345-9d5c-0251fbcc3000 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:07:42.135981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb793c0-3d05-4345-9d5c-0251fbcc3000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:07:42.135981 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-fbb793c0-3d05-4345-9d5c-0251fbcc3000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:07:42.140149 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb793c0-3d05-4345-9d5c-0251fbcc3000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:07:42.141580 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb793c0-3d05-4345-9d5c-0251fbcc3000 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:07:42.149660 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1454/2884] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:07:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:07:52.145923 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=26 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:07:55.200294 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:07:55.201885 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:07:55.290066 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4eb6f-9d85-4d34-9523-0ddfa6a8db68 admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:07:55.297045 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:07:55.297671 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:07:55.302626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:07:55.302626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:07:55.302626 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:07:55.313758 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:07:55.315386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:07:55.328840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1431/2885] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:07:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:07:57.171063 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a971d-87ec-41f5-a915-0e6ddab83243 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:07:57.171567 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a971d-87ec-41f5-a915-0e6ddab83243 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:07:57.171683 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a971d-87ec-41f5-a915-0e6ddab83243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:07:57.191105 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-6d1a971d-87ec-41f5-a915-0e6ddab83243 None None] MFA Rules not processed for user `c911a9324cb744a6a1aeb068565d3f89`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:07:57.192300 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6d1a971d-87ec-41f5-a915-0e6ddab83243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:07:57.248304 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1455/2886] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:07:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:07:57.559918 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:07:57.561604 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:07:57.573179 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82b0bf-1cb5-4401-9c88-d377dec7799c service nova] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:07:57.576785 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:07:57.577386 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:07:57.577661 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:07:57.577930 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:07:57.579624 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:07:57.590239 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:07:57.591688 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:07:57.599848 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1432/2887] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:07:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:07:59.384746 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a971d-87ec-41f5-a915-0e6ddab83243 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:07:59.386261 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-6d1a971d-87ec-41f5-a915-0e6ddab83243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:07:59.459421 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a971d-87ec-41f5-a915-0e6ddab83243 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:07:59.465353 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:07:59.465957 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:07:59.466231 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:07:59.466499 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:07:59.468241 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:07:59.480227 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:07:59.481807 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:07:59.491801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1456/2888] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:07:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:08:07.596157 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=40 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:08:09.487450 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=57 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:08:15.363289 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:08:15.364840 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:15.381738 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9155060-f1ae-4b32-873e-a45e404b5e62 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:08:15.388219 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '500d2f2bd83d499aa2814c6d7cc71795', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:08:15.388821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:08:15.389097 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:08:15.389367 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:08:15.391463 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:08:15.404113 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:08:15.405693 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:08:15.416633 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1433/2889] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:08:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:08:25.411934 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=6 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:09:52.386595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:09:52.388041 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:09:52.401136 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c482f-b921-4c65-be9e-ac0e55234b3c service placement] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:09:52.406272 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0aef7-75c3-4e7a-a55d-6ff06e5f7025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:09:52.406871 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0aef7-75c3-4e7a-a55d-6ff06e5f7025 service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:09:52.407177 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0aef7-75c3-4e7a-a55d-6ff06e5f7025 service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:09:52.407448 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0aef7-75c3-4e7a-a55d-6ff06e5f7025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:09:52.409168 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b3d0aef7-75c3-4e7a-a55d-6ff06e5f7025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:09:52.421320 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0aef7-75c3-4e7a-a55d-6ff06e5f7025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:09:52.422899 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0aef7-75c3-4e7a-a55d-6ff06e5f7025 service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:09:52.434275 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1457/2890] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:09:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:10:00.092421 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:10:00.093821 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:10:00.150846 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caed1ae-40df-412a-b624-200eca1310fd service nova] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:10:00.154420 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58567e28a5f741eda021aecab43d6a63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:10:00.155123 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:10:00.155396 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:10:00.155662 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:10:00.157360 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:10:00.168907 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:10:00.170362 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:10:00.180087 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1434/2891] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:10:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:10:00.289960 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31581dbf-ff9e-4227-a9e7-a813901a3e20 None None] REQUEST_METHOD: `POST` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:10:00.290490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31581dbf-ff9e-4227-a9e7-a813901a3e20 None None] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:10:00.290490 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31581dbf-ff9e-4227-a9e7-a813901a3e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:10:00.309530 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.auth.core [None req-31581dbf-ff9e-4227-a9e7-a813901a3e20 None None] MFA Rules not processed for user `c911a9324cb744a6a1aeb068565d3f89`. Rule list: `[]` (Enabled: `True`). {{(pid=88508) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 16:10:00.310721 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-31581dbf-ff9e-4227-a9e7-a813901a3e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:10:00.317480 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1458/2892] 10.222.0.94 () {58 vars in 958 bytes} [Fri Aug 26 16:10:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 16:10:10.175317 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=26 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:10:10.304509 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=9 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:11:51.012030 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:11:51.021832 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d273a8b-8e22-44d5-b238-997f0ed32ea8 service cinder] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:11:51.025605 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:11:51.026204 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:11:51.026505 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:11:51.026769 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:11:51.028540 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:11:51.042766 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:11:51.044221 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:11:51.056145 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1435/2893] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:11:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:11:52.248365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31581dbf-ff9e-4227-a9e7-a813901a3e20 None None] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:11:52.259012 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31581dbf-ff9e-4227-a9e7-a813901a3e20 None None] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:11:52.262577 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:11:52.263233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:11:52.263507 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:11:52.263772 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:11:52.265506 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:11:52.281518 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:11:52.283812 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:11:52.295637 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1459/2894] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:11:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:12:01.050817 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=6 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:12:02.290365 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=6 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:12:05.560409 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:12:05.570545 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa2f4-49b6-4fab-8d8c-a3b516c34707 service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:12:05.577538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ad758153334ec28940156d09495ae6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:12:05.577538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:12:05.577538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:12:05.577538 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:12:05.578712 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:12:05.641466 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:12:05.642910 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:12:05.652508 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1436/2895] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:12:09.195912 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:12:09.207890 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5821dd-58d1-4c17-b14a-ba567d7ff966 service neutron] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:12:09.213323 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:12:09.213936 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:12:09.214209 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:12:09.214477 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:12:09.216318 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:12:09.229785 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:12:09.231415 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:12:09.242373 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1460/2896] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:12:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:12:12.813288 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:12:12.824737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c697d-4965-425a-9ab9-2693924bc55d service neutron] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:12:12.830426 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:12:12.831007 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] REQUEST_METHOD: `GET` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:12:12.831293 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:12:12.831567 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:12:12.833344 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:12:12.845614 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:12:12.847174 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:12:12.857383 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1437/2897] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:12:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:12:12.939712 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:12:12.941349 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:12:12.955809 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cd931-a597-43d4-bfef-7339dedfcdb2 service placement] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:12:12.960396 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:12:12.961053 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:12:12.961326 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:12:12.961595 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] PATH_INFO: `/v3/users/c911a9324cb744a6a1aeb068565d3f89` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:12:12.962703 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c911a9324cb744a6a1aeb068565d3f89)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:12:12.964659 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:12:13.122934 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c911a9324cb744a6a1aeb068565d3f89'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.123614 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c911a9324cb744a6a1aeb068565d3f89'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.135503 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c911a9324cb744a6a1aeb068565d3f89'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.194628 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c911a9324cb744a6a1aeb068565d3f89'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.202160 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c911a9324cb744a6a1aeb068565d3f89'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.209246 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1461/2898] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:12:12 2022] DELETE /identity/v3/users/c911a9324cb744a6a1aeb068565d3f89 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:12:13.448731 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:12:13.466793 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45ff72-24f0-4a6c-acca-5d1120f0cbeb service placement] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:12:13.531525 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:12:13.532161 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:12:13.532527 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:12:13.532737 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] PATH_INFO: `/v3/projects/3c5f0e0c02e64047b9ca17a7320b0bf5` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:12:13.533992 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5f0e0c02e64047b9ca17a7320b0bf5)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:12:13.535953 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:12:13.618623 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5f0e0c02e64047b9ca17a7320b0bf5', 'request_id': 'req-f544e862-b66d-414c-8e30-0aa680d9389b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.624016 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5f0e0c02e64047b9ca17a7320b0bf5', 'request_id': 'req-f544e862-b66d-414c-8e30-0aa680d9389b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.669867 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5f0e0c02e64047b9ca17a7320b0bf5', 'request_id': 'req-f544e862-b66d-414c-8e30-0aa680d9389b'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.682602 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] The token cache is being invalidate because project 3c5f0e0c02e64047b9ca17a7320b0bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:12:13.689392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.689392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.689392 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1438/2899] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:12:13 2022] DELETE /identity/v3/projects/3c5f0e0c02e64047b9ca17a7320b0bf5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:12:13.689801 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:12:13.691277 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:12:13.704732 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485657d-9770-498b-a227-a5b3380dd06a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:12:13.765313 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:12:13.765954 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:12:13.766233 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:12:13.766505 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] PATH_INFO: `/v3/users/81f2329e50b347b89b6f9a980bea2f07` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:12:13.767778 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f2329e50b347b89b6f9a980bea2f07)` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:12:13.769705 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:12:13.986467 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f2329e50b347b89b6f9a980bea2f07'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.987199 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f2329e50b347b89b6f9a980bea2f07'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:13.997439 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:12:13.999713 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f2329e50b347b89b6f9a980bea2f07'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:14.098447 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f2329e50b347b89b6f9a980bea2f07'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:14.106306 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.notifications [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f2329e50b347b89b6f9a980bea2f07'} {{(pid=88508) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:14.113165 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1462/2900] 10.222.0.94 () {58 vars in 1224 bytes} [Fri Aug 26 16:12:13 2022] DELETE /identity/v3/users/81f2329e50b347b89b6f9a980bea2f07 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:12:14.374810 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] Authenticating user token {{(pid=88509) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:12:14.376954 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.fernet_utils [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88509) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:12:14.390663 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544e862-b66d-414c-8e30-0aa680d9389b admin admin] Validating token access rules against request {{(pid=88509) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:12:14.399302 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:12:14.449958 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f7a209666c0478690d711f7d361c03b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27572d94c00243fdbbfd37ae021fbe77', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88509) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:12:14.450599 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:12:14.450874 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] SCRIPT_NAME: `/identity` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:12:14.451214 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] PATH_INFO: `/v3/projects/d89e0264a19e4a79a8fe884c40a4f593` {{(pid=88509) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:12:14.452340 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d89e0264a19e4a79a8fe884c40a4f593)` {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:12:14.454740 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] RBAC: Authorization granted {{(pid=88509) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:12:14.567594 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd89e0264a19e4a79a8fe884c40a4f593', 'request_id': 'req-2ba463cb-3350-44f1-acf6-46561e91f489'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:14.574027 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd89e0264a19e4a79a8fe884c40a4f593', 'request_id': 'req-2ba463cb-3350-44f1-acf6-46561e91f489'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:14.647430 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd89e0264a19e4a79a8fe884c40a4f593', 'request_id': 'req-2ba463cb-3350-44f1-acf6-46561e91f489'} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:14.660429 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] The token cache is being invalidate because project d89e0264a19e4a79a8fe884c40a4f593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88509) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 16:12:14.661946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:14.661946 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG keystone.notifications [None req-2ba463cb-3350-44f1-acf6-46561e91f489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88509) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 16:12:14.662797 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: [pid: 88509|app: 0|req: 1439/2901] 10.222.0.94 () {58 vars in 1233 bytes} [Fri Aug 26 16:12:14 2022] DELETE /identity/v3/projects/d89e0264a19e4a79a8fe884c40a4f593 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 16:12:24.149836 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:12:24.656743 dv-cinder-851449-7-iscsi devstack@keystone.service[88509]: DEBUG dbcounter [-] [88509] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88509) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 16:14:59.349578 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] Authenticating user token {{(pid=88508) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 16:14:59.351043 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:14:59.489847 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc749e-b660-41bf-acf1-a891a3f2121a admin admin] Validating token access rules against request {{(pid=88508) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 16:14:59.550401 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71817b9-a280-4592-a3f4-1ca731e55dc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94983527f4f044b683921034aa1829b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5b63f7785db74906bb58fada43b9ff55', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88508) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 16:14:59.551031 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71817b9-a280-4592-a3f4-1ca731e55dc1 service placement] REQUEST_METHOD: `GET` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 16:14:59.551308 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71817b9-a280-4592-a3f4-1ca731e55dc1 service placement] SCRIPT_NAME: `/identity` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 16:14:59.551581 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71817b9-a280-4592-a3f4-1ca731e55dc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88508) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 16:14:59.553332 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.fernet_utils [None req-b71817b9-a280-4592-a3f4-1ca731e55dc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88508) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 16:14:59.566007 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71817b9-a280-4592-a3f4-1ca731e55dc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 16:14:59.567639 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71817b9-a280-4592-a3f4-1ca731e55dc1 service placement] RBAC: Authorization granted {{(pid=88508) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 16:14:59.577995 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: [pid: 88508|app: 0|req: 1463/2902] 10.222.0.94 () {60 vars in 1365 bytes} [Fri Aug 26 16:14:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 16:15:09.573599 dv-cinder-851449-7-iscsi devstack@keystone.service[88508]: DEBUG dbcounter [-] [88508] Writing DB stats keystone:SELECT=89 {{(pid=88508) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}